Smaller Election#

class smoothcrawler_cluster.election.SmallerElection[source]#

Election by index of crawler name

The criteria of this election strategy is the index which is the last characters in crawler’s name, e.g., sc-cluster_1. It would get all indexes from every crawler’s name and filter some of them which is/are smaller than the spot option value.

For example, it has 5 crawlers and its name from sc-cluster_1 to sc-cluster_5. If the option spot value is 3, it would mark the crawlers whose index of name is smaller than or equal to 3 as ElectionResult.WINNER; therefore , the winners of election are the crawlers sc-cluster_1 to sc-cluster_3, they would be Runner. And the rest would be the Backup_Runner.

elect(candidate: str, member: List[str], index_sep: str, spot: int) → ElectionResult[source]#

Run the election processing to verify who is/are Runner and otherwise is/are Backup_Runner finally.

Parameters:

  • candidate (str) – The crawler instance which apply for the runner election. In generally, it is the crawler instance’s name.

  • member (list of str) – All the instances which apply for the runner election.

  • index_sep (str) – The separation of index in crawler instance’s name.

  • spot (int) – The amount of Winner, in the other words, Runner it could have.

Returns:

Final election result.

Return type:

ElectionResult

parse_index(member: List[str], index_sep: str) → List[int][source]#

Parse all the member’s identity from their name.

Parameters:

  • member (List[str]) – All the members who would participate in this election. In generally, the members would be the value of meta-data GroupSate.current_crawler.

  • index_sep (str) – The separation which could separate the crawler instance’s name and get the identity from it.

Returns:

A list of int object which are the identities from all members in cluster.

filter(member_indexes: List[int], candidate: str, index_sep: str, spot: int) → List[bool][source]#

Filter all member’s identity and generate winners who would be RUNNER in cluster, nor they would be BACKUP RUNNER.

Parameters:

  • member_indexes (List[int]) – A list of int object which are the identities from all members in cluster.

  • candidate (str) – The current crawler instance who participates in this election.

  • index_sep (str) – The separation which could separate the crawler instance’s name and get the identity from it.

  • spot (int) – The amount of Winner, in the other words, Runner it could have.

Returns:

A list of bool object which means the current crawler instance is winner.