Shulou(Shulou.com)06/03 Report--

Easy Scheduler Release 1.0.2

Easy Scheduler 1.0.2 is the third version of the 1.x series. This version adds an open API for scheduling, worker grouping (the machine group that specifies the task to run), task flow and service monitoring, and support for oracle, clickhouse, and so on, as follows:

New feature: [EasyScheduler-79] scheduling opens the interface to the public through token You can operate through api [EasyScheduler-138] you can specify the machine (group) [EasyScheduler-139] task flow monitoring and Master, Worker, Zookeeper running status monitoring [EasyScheduler-140] workflow definition-add process timeout alarm [EasyScheduler-134] task types support Oracle, CLICKHOUSE, SQLSERVER, Sql task node can independently select CC mail users [EasyScheduler-141] user management-users can bind queues The user queue level is higher than the tenant queue level If the user queue is empty, look for tenant queue enhancement: [EasyScheduler-154] tenant coding allows coding fixes such as pure digits or underscores:

[EasyScheduler-135] Python task can specify python version

[EasyScheduler-125] the mobile number in the user's account cannot recognize the beginning of the latest number 166of China Unicom.

[EasyScheduler-178] fixed subtle spelling errors in ProcessDao

[EasyScheduler-129] in tenant management, special characters such as tenant codes with underscores cannot pass the verification thanks:

Last but not least, there can be no new version without the contributions of the following partners:

Baoqi, chubbyjiang, coreychen, chgxtony, cmdares, datuzi, dingchao, fanguanqun, Fengqingyang, gaojun416, googlechorme, hyperknob, hujiang75277381, huanzui, kinssun, ivivi727, jimmy, jiangzhx, kevin5210, lidongdai, lshmouse, lenboo, lyf198972, lgcareer, lzy305, moranrr, millionfor, mazhong8808, programlief, qiaozhanwei, roy110, swxchappy, sherlock111, samz406, swxchappy, qq389401879, lzy305, vkingnew, William-GuoWei, woniulinux, woniulinux,

And many enthusiastic friends in WeChat group! Thank you very much!

Easy Scheduler introduction

1. Background

With the efforts of many technology partners, after nearly 2 years of research and development iteration, internal business divestiture and refactoring, and after a trial period by a group of seed users, EasyScheduler finally ushered in its first official open source release-1.0.0.

It is believed that partners who have done data processing should be familiar with open source scheduling systems such as oozie, azkaban and airflow. In using these scheduling systems, they may have such experiences: for example, the configuration workflow task cannot be visualized, the running status of the task cannot be viewed online in real time,

The task cannot be paused while running, cannot support parameter passing, cannot be replenished, cannot be used by multi-tenants, and the scheduling system is not highly available, and so on. It is under this background that Easy Scheduler came into being, and its goal is to make scheduling more easy, and we can see our original intention from its Chinese name "easy scheduling".

2. Design features

Easy Scheduler is a distributed workflow task scheduling system, which mainly solves all kinds of problems caused by the complex dependency relationship of data research and development ETL.

Its main objectives are as follows:

By correlating Task according to task dependency in the way of DAG diagram, you can visually monitor the running status of tasks in real time and support rich task types: Shell, MR, Spark, SQL (mysql, postgresql, hive, sparksql), Python,Sub_Process, Procedure, etc. support workflow timing scheduling, dependent scheduling, manual scheduling, manual pause / stop / resume. At the same time, support for failure retry / alarm, recovery failure from specified nodes, Kill tasks and other operations to support workflow priority, task priority and task failover, and task timeout alarm / failure support workflow global parameters and node custom parameter settings support online upload / download of resource files, management, etc. Support for online file creation, editing, online viewing and scrolling of task logs, online downloading of logs and so on to achieve cluster HA, Master cluster and Worker cluster decentralization support for Master/Worker cpu load,memory through Zookeeper Cpu online view support workflow running history tree / Gantt chart display, support task status statistics, process status statistics support complement support multi-tenant support internationalization and more partners to explore 4, comparison with similar scheduling systems

5. System partial screenshot

6. Easy Scheduler back-end deployment document Easy Scheduler front-end deployment document Easy Scheduler user manual

For more documentation, please refer to:

Easy Scheduler Chinese online documents

7. Thank you

Easy Scheduler uses many excellent open source projects, such as google's guava, guice, grpc,netty,ali 's bonecp,quartz, and apache's many open source projects. We are also very grateful to oozie, azkaban, airflow and other excellent scheduling works for their inspiration.

It is because of standing on the shoulders of these open source projects that the birth of Easy Scheduler is possible. We are very grateful to all the open source software used for this! We also hope that we can not only benefit from open source, but also become open source.

Contributor, so we decided to contribute to easy scheduling and committed to long-term maintenance. I also hope that partners who have the same enthusiasm and belief in open source will join us to contribute to open source!

8. Postscript

Just a week after Easy Scheduler officially opened up on March 28, 2019.028, we felt the great enthusiasm of our partners for Easy Scheduler. Many partners proposed to use feedback, and some partners directly found the corresponding source code to ask questions or give better suggestions, or even rolled up their sleeves to write code on Easy Scheduler.

This is a great spiritual encouragement to our current major developers. Thank you very much for your enthusiasm and trust in us. We will continue to work with you on the road of making the dispatching system available out of the box. To make "data capabilities civilian", to contribute their passion and sweat to the data era!

Open source github address: https://github.com/analysys/EasyScheduler

Online document address: https://analysys.github.io/easyscheduler_docs_cn

Contribution code

You are very welcome to contribute the code. Please refer to the process of submitting the code:

Https://github.com/analysys/EasyScheduler/blob/master/CONTRIBUTING.md

Thank you.

Easy Scheduler uses many excellent open source projects, such as google's guava, guice, grpc,netty,ali 's bonecp,quartz, and apache's many open source projects, etc.

It is because of standing on the shoulders of these open source projects that the birth of Easy Scheduler is possible. We are very grateful to all the open source software used for this! We also hope that we can not only benefit from open source, but also become open source.

Contributor, so we decided to contribute to easy scheduling and committed to long-term maintenance. I also hope that partners who have the same enthusiasm and belief in open source will join us to contribute to open source!

Help

The fastest way to get response from our developers is to submit issues, or add our wechat: 510570367

In addition, we have deployed a stand-alone version of small demo. For those who want to experience it, please visit http://52.82.13.76:8888 Welcome contact experience (wechat: 510570367), which involves multiple modifications. Administrator login is not open for the time being! Suggestions are welcome!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.