Get the App
SLTechnology News&Howtos  ›  Internet Technology  › 

How to read external files with flinksql table

Shulou Source: shulou.com Published: 2022-06-02 06:41:18 09月14日 Update

This article mainly introduces "how to read external files from flinksql table". In daily operation, I believe many people have doubts about how to read external files from flinksql table. The editor consulted all kinds of data and sorted out simple and easy-to-use methods of operation. I hope it will be helpful to answer the doubts about "how to read external files from flinksql table". Next, please follow the editor to study!

1. TableEnvironment can register the directory Catalog, which can be based on the catalog registry

2. The table is specified by an identifier and consists of three parts: catalog, database name, and object name.

3. The table can be regular or virtual (view)

4. General tables can generally be used to describe external data, such as files, databases or message queues, or they can be converted directly from datastream.

5. Views can be created from existing tables, usually a result set of table api or sql queries

Code:

Package com.jd.data;import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;import org.apache.flink.table.api.DataTypes;import org.apache.flink.table.api.Table;import org.apache.flink.table.api.bridge.java.StreamTableEnvironment;import org.apache.flink.table.descriptors.FileSystem;import org.apache.flink.table.descriptors.OldCsv;import org.apache.flink.table.descriptors.Schema;import org.apache.flink.types.Row;import scala.Tuple3 Public class TableAipDemo03 {public static void main (String [] args) throws Exception {StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment (); env.setParallelism (1); / / 1. Creation table execution StreamTableEnvironment tableEnv = StreamTableEnvironment.create (env); String path = "/ Users/liuhaijing/Desktop/flinktestword/aaa.txt" TableEnv.connect (new FileSystem (). Path (path)) .withFormat (new OldCsv ()) / defines the formatting method .withSchema (new Schema (). Field ("a", DataTypes.STRING ()) / / defines the structure of the table .field ("b", DataTypes.STRING ()) .field ("c") DataTypes.STRING ()) .inAppendMode () .createTemporaryTable ("xxx") Table xxx = tableEnv.from ("xxx"); xxx.printSchema (); tableEnv.toAppendStream (xxx, Row.class). Print (); env.execute ("job");} at this point, the study of "how the flinksql table reads external files" is over, hoping to solve everyone's doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Tags: Files data learning general databases methods more views help utility next code objects articles are made up of identifiers identifiers formats registries messages Apple Docker Huawei Linux macOS MariaDB Microsoft MySQL NVidia OPPO Reno MariaDB Apple OPPO Reno Shulou Tech Info MySQL