Network Security Internet Technology Development Database Servers Mobile Phone Android Software Apple Software Computer Software News IT Information

In addition to Weibo, there is also WeChat

Please pay attention

WeChat public account

Shulou

Detailed explanation of the steps for Flume to listen for the increment of oracle table

2025-01-30 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Database >

Share

Shulou(Shulou.com)06/01 Report--

Requirements: get the incremental information of oracle table, send it to udp514 port, support ip configuration

Steps:

(1) odbc5.jar of the required jar oracle (look under the oracle installation directory / jdbc/lib)

(2) the latest small version of flume's open source package flume-ng-sql-source-1.4.3.jar seems to be 1.5. I can't remember it. This download address is available directly on csdn.

Both jar are copied to flume's lib.

(3) flume configuration file

A1.sources = r1a1.sinks = k1a1.channels = syslog receive configuration # a1.sources.r1.type = syslogudp#a1.sources.r1.port = 514#a1.sources.r1.host = 0.0.0.0#sql source#a1.sources.r1.type = org.keedio.flume.source.SQLSourcea1.sources.r1.hibernate.connection.url = jdbc:oracle:thin:@ 10.9.224.36:1521/soc# Hibernate Database connection propertiesa1.sources.r1.hibernate.connection.user = soc_appa1.sources.r1.hibernate.connection.password = soc_appa1.sources.r1.hibernate.connection.autocommit = truea1.sources.r1.hibernate.dialect = org.hibernate.dialect.Oracle10gDialecta1.sources.r1.hibernate.connection.driver_class = oracle.jdbc.driver.OracleDrivera1.sources.r1.run.query.delay=1a1.sources.r1.status.file.path = / usr/local/flumea1. Sources.r1.status.file.name = a1.sqlSource.status# Custom querya1.sources.r1.start.from = '2017-07-31 07:06:20'a1.sources.r1.custom.query = SELECT CHR (39) | | TO_CHAR (CREATETIME) 'YYYY-MM-DD HH24:MI:SS') | | CHR (39), MSG FROM FLUME_NG_SQL_SOURCE WHERE CREATETIME > TO_DATE ($@ $) 'YYYY-MM-DD HH24:MI:SS') ORDER BY CREATETIME ASCa1.sources.r1.batch.size = 6000a1.sources.r1.max.rows = 1000a1.sources.r1.hibernate.connection.provider_class = org.hibernate.connection.C3P0ConnectionProvidera1.sources.r1.hibernate.c3p0.min_size=1a1.sources.r1.hibernate.c3p0.max_size=10#sql source# Custom Type a1.sinks .k1.type = com.neusoft.utils.SyslogSink# configuration sends the ipa1.sinks.k1.destination = 10.9.224.36a1.channels.c1.type = memorya1.channels.c1.capacity = 10000a1.channels.c1.transactionCapacity = 10000a1.sources.r1.channels = c1a1.sinks.k1.channel = flume-ng agent startup command. / flume-ng agent-c.. / conf-f.. / conf/flume-sql.conf-n A1-Dflume.root.logger=INFO Console &

Since the self-defined sink is also mentioned in my blog, I configured a custom sink.

Summary

The above is the detailed explanation of the incremental steps of Flume monitoring oracle table introduced by the editor. I hope it will be helpful to you. If you have any questions, please leave me a message and the editor will reply to you in time. Thank you very much for your support to the website!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.

Views: 0

*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.

Share To

Database

Wechat

© 2024 shulou.com SLNews company. All rights reserved.

12
Report