In addition to Weibo, there is also WeChat
Please pay attention
WeChat public account
Shulou
2025-01-30 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Database >
Share
Shulou(Shulou.com)06/01 Report--
Requirements: get the incremental information of oracle table, send it to udp514 port, support ip configuration
Steps:
(1) odbc5.jar of the required jar oracle (look under the oracle installation directory / jdbc/lib)
(2) the latest small version of flume's open source package flume-ng-sql-source-1.4.3.jar seems to be 1.5. I can't remember it. This download address is available directly on csdn.
Both jar are copied to flume's lib.
(3) flume configuration file
A1.sources = r1a1.sinks = k1a1.channels = syslog receive configuration # a1.sources.r1.type = syslogudp#a1.sources.r1.port = 514#a1.sources.r1.host = 0.0.0.0#sql source#a1.sources.r1.type = org.keedio.flume.source.SQLSourcea1.sources.r1.hibernate.connection.url = jdbc:oracle:thin:@ 10.9.224.36:1521/soc# Hibernate Database connection propertiesa1.sources.r1.hibernate.connection.user = soc_appa1.sources.r1.hibernate.connection.password = soc_appa1.sources.r1.hibernate.connection.autocommit = truea1.sources.r1.hibernate.dialect = org.hibernate.dialect.Oracle10gDialecta1.sources.r1.hibernate.connection.driver_class = oracle.jdbc.driver.OracleDrivera1.sources.r1.run.query.delay=1a1.sources.r1.status.file.path = / usr/local/flumea1. Sources.r1.status.file.name = a1.sqlSource.status# Custom querya1.sources.r1.start.from = '2017-07-31 07:06:20'a1.sources.r1.custom.query = SELECT CHR (39) | | TO_CHAR (CREATETIME) 'YYYY-MM-DD HH24:MI:SS') | | CHR (39), MSG FROM FLUME_NG_SQL_SOURCE WHERE CREATETIME > TO_DATE ($@ $) 'YYYY-MM-DD HH24:MI:SS') ORDER BY CREATETIME ASCa1.sources.r1.batch.size = 6000a1.sources.r1.max.rows = 1000a1.sources.r1.hibernate.connection.provider_class = org.hibernate.connection.C3P0ConnectionProvidera1.sources.r1.hibernate.c3p0.min_size=1a1.sources.r1.hibernate.c3p0.max_size=10#sql source# Custom Type a1.sinks .k1.type = com.neusoft.utils.SyslogSink# configuration sends the ipa1.sinks.k1.destination = 10.9.224.36a1.channels.c1.type = memorya1.channels.c1.capacity = 10000a1.channels.c1.transactionCapacity = 10000a1.sources.r1.channels = c1a1.sinks.k1.channel = flume-ng agent startup command. / flume-ng agent-c.. / conf-f.. / conf/flume-sql.conf-n A1-Dflume.root.logger=INFO Console &
Since the self-defined sink is also mentioned in my blog, I configured a custom sink.
Summary
The above is the detailed explanation of the incremental steps of Flume monitoring oracle table introduced by the editor. I hope it will be helpful to you. If you have any questions, please leave me a message and the editor will reply to you in time. Thank you very much for your support to the website!
Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.
Views: 0
*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.
Continue with the installation of the previous hadoop.First, install zookooper1. Decompress zookoope
"Every 5-10 years, there's a rare product, a really special, very unusual product that's the most un
© 2024 shulou.com SLNews company. All rights reserved.