Contents Menu Expand Light mode Dark mode Auto light/dark mode
SmoothCrawler-Cluster 0.2.0 documentation
SmoothCrawler-Cluster 0.2.0 documentation

Contents:

  • Introduce
  • Installation
  • Quickly Start
  • Advanced Usage
    • Self-Define Utils
      • How to implement new Zookeeper client
      • How to write new Converter
    • How to customize new Election

Usage Guides:

  • No Leader
    • All are Runner
    • Some are Runner
  • Has Leader
    • Leader (Master)
    • Follower (Slave)
  • Handle Meta-Data
    • Depend on third party application - Zookeeper
    • Handle by itself

API Reference:

  • Crawler Sub-Package
    • Crawler Module
      • Abstract Crawlers
      • Zookeeper Crawlers
    • Attributes
    • Adapter
    • Dispatcher
    • Workflow
  • Register
    • General Registration
  • Election
    • Abstract Election
    • Smaller Election
  • Model
    • Meta-Data Modules
      • Abstract Meta-Data
      • Group State
      • Node State
      • Task
      • Heartbeat
    • Meta-Data Enum Objects
      • CrawlerRole
      • TaskState
      • HeartState
    • Inner Data
  • Inner Modules
    • Utils - Zookeeper
    • Utils - Converter

Development Documentation

  • Flow
    • Work Flow
    • Data Flow
  • Software Architecture
    • Crawler Sub-Package
      • Crawler
      • Workflow
      • Dispatcher
      • Attributes
    • Election
    • Utils
  • CI / CD

Change Logs

  • SmoothCrawler-Cluster 0.X
    • Version 0.X
Back to top
Edit this page

Software Architecture#

An image could be very helpful for everyone to quickly understand a relationship between many classes, it still works if the classes relationship is complex. This section has each one module’s images like UML for show and explain you their software architecture.

Following are the details of all current modules:

  • Crawler Sub-Package
    • Crawler
    • Workflow
    • Dispatcher
    • Attributes
  • Election
    • UML
    • Description
  • Utils
    • Zookeeper
    • Converter
Next
Crawler Sub-Package
Previous
Data Flow
Copyright © 2022, Liu, Bryant
Made with Furo