Shulou(Shulou.com)06/02 Report--

As the beneficiaries of open source software, while enjoying the technical convenience brought by open source, we also actively embrace open source and give back to open source. As the company's 16th open source project, the Urban Transport Index (TTI) has released desensitized data through the Gaia program. Downloaders are distributed in 127 universities or scientific research institutions, covering 70% of the double first-tier universities.

Open source software is often seen in map data processing, and the following are common:

GDAL (Geospatial Data Abstraction Library)

GDAL is a spatial raster data conversion library under the X/MIT license agreement. OGR is a branch of the GDAL project that provides support for vector data. Raster data and vector data are two common data formats in map data. Popular understanding is that raster data is expressed by pixels and vector data is expressed by coordinate points. Common raster data include remote sensing images, scanned maps and so on. Common vector data include all kinds of point, line and surface data, such as POI, road network, river system or lake. GDAL can easily read and write raster or vector data.

Sample code for GDAL to read remote sensing images:

GDALDataset* pDataSet = (GDALDataset*) GDALOpen ("/ Users/didi/Desktop/test.img", GA_ReadOnly); / / Affine transformation 6 parameters double geoTransform [6] = {0}; pDataSet- > GetGeoTransform (geoTransform); / / Image width int nWidth = pDataSet- > GetRasterXSize (); / / Image height int nHeight = pDataSet- > GetRasterYSize (); / / Pixel value Matrix unsigned char* pPixelValue = (unsigned char*) malloc (sizeof (unsigned char) * nWidth * nHeight); memset (pPixelValue,0,nWidth * nHeight) CPLErr err = pDataSet- > RasterIO (GF_Read,0,0,nWidth,nHeight,pPixelValue,nWidth,nHeight,GDT_Byte,1,NULL,0,0,0); free (pPixelValue); GDALClose (pDataSet)

There are some practical image processing algorithms in GDAL, such as GDALSimpleSURF class can realize feature point detection and matching, GDALSieveFilter can delete 4-connected or 8-connected pixel polygons (connected regions with the same pixel value), such as trachoma noise. As shown in the following figure

Figure 1 before processing

Figure 2 after processing

Sample code for OGR to read vector data:

GDALDataset* pDS = (GDALDataset*) GDALOpenEx ("/ Users/didi/Desktop/test.shp", GDAL_OF_VECTOR | GDAL_OF_READONLY, NULL, NULL, NULL); OGRLayer* pLayer = pDS- > GetLayer (0); OGRFeature * pFeature = NULL;pLayer- > ResetReading (); while ((pFeature = pLayer- > GetNextFeature ())! = NULL) {/ / get geometric information OGRGeometry* pGeom = pFeature- > GetGeometryRef (); / / get the value of the field Length double dfLength = pFeature- > GetFieldAsDouble ("Length") OGRFeature::DestroyFeature (pFeature);} / / close dataset GDALClose (pDS); GEOS (Geometry Engine Open Source)

GEOS is a very classical spatial topological relation operation library, and the performance of some algorithms is not inferior to that of commercial GIS software. After reading the geometric information of vector data, GDAL needs the support of GEOS in spatial operation or topology judgment. In general, when compiling the GDAL source code, the GEOS is compiled directly, and the geometry classes in GDAL can also be converted into GEOS geometry classes, so that you can directly use the GEOS library for data processing. Using GEOS, you can easily calculate the difference, intersection, symmetry difference and buffer of two geometric objects. Sometimes we need to merge multiple adjacent polygons into a single polygon, the conventional usage is to use the union method to merge them, when the number of polygons to be merged is large, the efficiency will be very low, here we can use the method of calculating the buffer to deal with, the efficiency will be greatly improved.

Figure 3 polygons to be merged

Fig. 4 merge result chart

Sample code:

/ / Blue polygon char* szWKT_1 = "POLYGON ((113.885 22.6815, 113.9425 22.6585, 113.91 22.7, 113.885 22.6815)"; / / Orange polygon char* szWKT_2 = "POLYGON ((113.91 22.7, 113.9425 22.6585, 113.9675 22.689, 113.91 22.7)"; OGRGeometry* pGeom_1 = NULL;OGRGeometry* pGeom_2 = NULL;OGRGeometryFactory::createFromWkt (& szWKT_1, NULL, & pGeom_1) OGRGeometryFactory::createFromWkt (& szWKT_2, NULL, & pGeom_2); OGRMultiPolygon* pMultiPolygon = (OGRMultiPolygon*) OGRGeometryFactory::createGeometry (wkbMultiPolygon); pMultiPolygon- > addGeometryDirectly (pGeom_1); pMultiPolygon- > addGeometryDirectly (pGeom_2); / / replace Union with Buffer, buffer distance is set to 0//pUnion as purple polygon OGRGeometry* pUnion = pMultiPolygon- > Buffer (0); PROJ.4

Proj.4 is the most famous map projection library in open source GIS, and many GIS open source software projects directly use Proj.4 library files. The main functions are the transformation of projection coordinates and geographical coordinates, the conversion of coordinate system, the transformation of datum and so on. The WGS84 coordinate system is the most commonly used coordinate system, and the GPS track point is the geographical coordinate system. Sometimes we need to do some mathematical operations based on the projection coordinate system, that is, Gauss positive calculation. Such as length calculation. Here, we take a link in the area with 3 °zoning and 117 °central longitude to illustrate:

Fig. 5 3-degree zoning chart

Siwei_id:9000025617988

Geographical coordinates: LINESTRING (116.9710037 36.6434771116.97049 36.64324116.97014 36.64304116.96921 36.64244116.96787 36.64177)

Projection coordinates: LINESTRING (497406.947036178 4057018.36084719) 497361.0002820454056992.06317016Magi 497329.693821681 4056969.87826856) 497246.504857284 4056903.32080414 497126.64614413 4056829.00824338). Under the projection coordinates, we can directly use the distance formula between two points to calculate the length of this link, length=339 meters.

LibSpatialindex

Libspatialindex is an efficient spatial index library for C++. Support complex queries, such as range query, point location query, nearest neighbor query, K neighbor query and parameterized query. Sample code for creating a memory space index:

IStorageManager* diskfile = StorageManager::createNewMemoryStorageManager (); StorageManager::IBuffer* file = StorageManager::createNewRandomEvictionsBuffer (* diskfile, 10, false); double fillFactor = 0.7 × uint32int32Secrett indexCapacity = 10mituint32pragt leafCapacity = 10tint32intt dimension = 2TrentRTreeVariant variant = RTree::RV_RSTAR; id_type indexIdentifier;ISpatialIndex* tree = RTree::createNewRTree (* file, fillFactor, indexCapacity, leafCapacity, dimension, variant, indexIdentifier); x1 = enve.MinX;y1 = enve.MinY;x2 = enve.MaxX;y2 = enve.MaxY Plow [0] = x1Tracplow [1] = y1Tracphigh [0] = x2delete tree;delete file;delete diskfile;SpatiaLite phigh [1] = y2There is the circumscribed rectangle of the geometric object, and id is the unique identifier of the long integer type Region r = Region (plow, phigh, 2); tree- > insertData (0,0, r, id); delete tree;delete file;delete diskfile;SpatiaLite

SQLite is a very small relational database, which is often integrated in various mobile applications. In order to expand the spatial data storage function of SQLite, based on the kernel of SQLite, the spatial SQL function is added. SpatiaLite uses RTree as the spatial index. After creating a spatial index on a table, it can carry out efficient spatial query processing. Sample code for creating a spatial index: select CreateSpatialIndex ('table_name',' column_name')

PCL (Point Cloud Library)

Laser point cloud is a kind of map vector data which is widely used in recent years, which can efficiently obtain the three-dimensional coordinates of the target. Because of its high price of supporting equipment, only a few graphic merchants can afford it. For 3D point cloud processing, PCL is a modular C++ template library, which realizes point cloud acquisition, filtering, segmentation, registration, retrieval, feature extraction, recognition, tracking, surface reconstruction, visualization and so on based on the third party library.

PostGIS

PostGIS is just a plug-in for PostgreSQL, but it turns PostgreSQL into a powerful spatial database. PostGIS has excellent spatial query performance. If spatial data storage tools such as Spatialite and libspatialindex are warships, then PostGIS is an aircraft carrier. Among them, pgRouting adds routing function and realizes the classical algorithms in navigation path planning, such as Dijkstra algorithm, A* algorithm and traveling Salesman algorithm. After sorting the map data into a format that meets its specifications, you can quickly create a simple path planning and calculation service. In spatial query, PostGIS can achieve the effect of ready-to-check. In figure 6, if you want to obtain the road network data in the orange polygon, the ordinary spatial query result is a set of green query results. If you want to obtain the data in the polygon, you need to make a secondary spatial topology judgment in the query result set. In PostGIS, because of its unique GiST spatial index, a spatial query can be done, which greatly improves the query efficiency.

Fig. 6 effect diagram of spatial query

GeoServer

GeoServer is the J2EE implementation of OpenGIS Web server specification. GeoServer can be used to publish map data conveniently, allowing users to update, delete and insert feature data. Its data source supports PostGIS, MySQL, GeoMesa and so on. Users can customize the map drawing style and make various thematic maps by editing the SLD file. The following figure shows a schematic diagram of publishing a WMS service using different SLD files.

Fig. 7 track chart

Fig. 8 trajectory thermal map

OpenLayer 、 Mapbox

OpenLayer and MapBox application scenarios are similar. The JavaScript class library package provided for Web GIS client development is used to access map data published in standard format. Based on OpenLayer and MapBox, the front-end map can be displayed more beautifully.

QGIS

QGIS is a free, open source, cross-platform (Linux/Windows/Mac/Android) GIS desktop software based on QT C++ development. It provides powerful functions of spatial data display, editing and analysis. QGIS and many open source projects, use CMake to compile, because QGIS integrates most of the third-party software, based on the source code to compile executable QGIS, which is a test of patience, just like building your own building, each brick needs to be built by yourself. After compiling the Debug version of QGIS, you can debug its source code and carry out secondary development based on SDK. The design and architecture of QGIS is very exquisite and worth learning.

Mapnik

Mapnik is an open source map rendering engine, it can provide spatial data calculation and visualization services for a variety of data sources, including PostGIS,Shapefile,Geojson,SQLite, including png tiles, vector tiles, and it supports custom rendering style configuration with high flexibility. The Open Street Map master layer is rendered with Mapnik.

GRASS GIS

In the geographic information system industry, if ArcGIS (windows in the operating system line) is Qiao Feng in closed source software, GRASS GIS is Murong Fu in open source software. GRASS can deal with vector and image data, and carry out spatio-temporal analysis, spatial modeling, spatial analysis, map visualization and other operations. Some image processing algorithms in GRASS are comparable to professional commercial software in terms of performance and effectiveness. Although GRASS provides some visual operation interfaces, most of the functions need to interact with commands. For ordinary users, it is difficult to use, so it is more suitable for researchers or university teachers.

JTS

For the Java version of GEOS, please refer to the GEOS section. In the road network operation, you will encounter the situation of merging multiple roads connected from end to end into a single road. You can do this very well by using the LineMerger class in JTS. Sample code:

WKTReader reader = new WKTReader (); Geometry geom_1 = reader.read ("LINESTRING (116.9632000000000562 36.6488200000000662, 116.6849000000274 36.64882000000000062)"); Geometry geom_2 = reader.read ("LINESTRING (116.68490000000274 36.64882000000006616686200000000636820000000137 36.6488200000000648820000000000616616620000000136.648820000000000616.68779999999531 36.64880999999999944)"); LineMerger lineMerger = new LineMerger () / / you don't need to add geometric objects in order, as long as the coordinate points at the beginning and end of the road coincide, you can lineMerger.add (geom_1); lineMerger.add (geom_2); lineMerger.add (geom_3); Collection mergedLineStrings = lineMerger.getMergedLineStrings (); System.out.println (mergedLineStrings.toString ()); print result: [LINESTRING (116.96832 36.64882, 116.96849 36.64882, 116.96862 36.64882, 116.96878 36.64881)] GeoTools

The function is similar to GDAL, and the spatial topology algorithm is implemented by JTS.

GeoMesa

GeoMesa is an open source toolkit for spatio-temporal data processing, which can support geographic information analysis and distributed computing in big data scenarios. Can be better compatible with big data processing framework, such as HBase, Spark, big data Architecture Department of the company provides a complete GeoMesa solution, you can refer to the relevant information of the intranet

Author: Zhang Shenzhen

If you register Didi Yun now, you can get a 30-yuan Didi coupon without a threshold.

50% discount for newly purchased cloud services in January, 4.5% discount in March and 60% discount in June.

Didi Yun messenger recruits, recommending up to 50% rebate

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.