Shulou(Shulou.com)06/03 Report--

This article mainly explains "what is the dynamic library of C language". The content of the explanation in this article is simple and clear, and it is easy to learn and understand. Let's follow the editor's train of thought to study and learn what is the dynamic library of C language.

What? There is a high-performance base64 library lying here?

Let's take this repo as an example: https:// github.com/aklomp/base64. This is a Base64 encoding / decoding library written in C, and supports SIMD.

You can simply run the library's benchmark:

Karminski@router02:/data/works/base64 $make clean & & SSSE3_CFLAGS=-mssse3 AVX2_CFLAGS=-mavx2 make & & make-C test. Testing with buffer size 100 KB, fastest of 10 * 100 AVX2 encode 12718.47 MB/sec AVX2 decode 14542.81 MB/sec plain encode 3657.40 MB/sec plain decode 3433.23 MB/sec SSSE3 encode 7269.55 MB/sec SSSE3 decode 8173.10 MB/sec...

My CPU is Intel (R) Xeon (R) CPU E3-1246 v3 @ 3.50GHz. You can see that CPU can reach more than 12GB/s if it supports AVX2. This performance is so powerful that it can't even keep up with ordinary SSD.

The first step we need is to compile the repo into a dynamic library. But the repo does not provide a compilation option for the dynamic library, so we changed the Makefile of the project.

CFLAGS + =-std=c99-O3-Wall-Wextra-pedantic # Set OBJCOPY if not defined by environment: OBJCOPY? = objcopy OBJS =\ lib/arch/avx2/codec.o\ lib/arch/generic/codec.o\ lib/arch/neon32/codec.o\ lib/arch/neon64/codec.o\ lib/arch/ssse3/codec.o\ lib/arch/sse41/codec.o\ lib/arch/sse42/codec.o\ lib/arch/avx/codec.o\ lib/ Lib.o\ lib/codec_choose.o\ lib/tables/tables.o SOOBJS =\ lib/arch/avx2/codec.so\ lib/arch/generic/codec.so\ lib/arch/neon32/codec.so\ lib/arch/neon64/codec.so\ lib/arch/ssse3/codec.so\ lib/arch/sse41/codec.so\ lib/arch/sse42/codec.so\ lib/arch/avx/codec.so\ lib/lib.so\ Lib/codec_choose.so\ lib/tables/tables.so HAVE_AVX2 = 0 HAVE_NEON32 = 0 HAVE_NEON64 = 0 HAVE_SSSE3 = 0 HAVE_SSE41 = 0 HAVE_SSE42 = 0 HAVE_AVX = 0 # The user should supply compiler flags for the codecs they want to build. # Check which codecs we're going to include: ifdef AVX2_CFLAGS HAVE_AVX2 = 1 endif ifdef NEON32_CFLAGS HAVE_NEON32 = 1 endif ifdef NEON64_CFLAGS HAVE_NEON64 = 1 endif ifdef SSSE3_CFLAGS HAVE_SSSE3 = 1 endif ifdef SSE41_CFLAGS HAVE_SSE41 = 1 endif ifdef SSE42_CFLAGS HAVE_SSE42 = 1 endif ifdef AVX_CFLAGS HAVE_AVX = 1 endif ifdef OPENMP CFLAGS + =-fopenmp endif .PHONY: all analyze clean all: bin/base64 lib/libbase64.o lib/libbase64.so bin/ Base64: bin/base64.o lib/libbase64.o lib/libbase64.so $(CC) $(CFLAGS)-o $@ $^ lib/libbase64.o: $(OBJS) $(LD)-r-o $@ $^ $(OBJCOPY)-keep-global-symbols=lib/exports.txt $@ lib/libbase64.so: $(SOOBJS) $(LD)-shared-fPIC-o $@ $^ $(OBJCOPY)-keep-global-symbols=lib/exports .txt $@ lib/config.h: @ echo "# define HAVE_AVX2 $(HAVE_AVX2)" > $@ @ echo "# define HAVE_NEON32 $(HAVE_NEON32)" > > $@ @ echo "# define HAVE_NEON64 $(HAVE_NEON64)" > > $@ @ echo "# define HAVE_SSSE3 $(HAVE_SSSE3)" > > $@ @ echo "# define HAVE_SSE41 $(HAVE_SSE41)" > > $ @ @ echo "# define HAVE_SSE42 $(HAVE_SSE42)" > > $@ @ echo "# define HAVE_AVX $(HAVE_AVX)" > > $@ $(OBJS): lib/config.h $(SOOBJS): lib/config.h # o lib/arch/avx2/codec.o: CFLAGS + = $(AVX2_CFLAGS) lib/arch/neon32/codec.o: CFLAGS + = $(NEON32_CFLAGS) lib/arch/neon64/codec.o: CFLAGS + = $(NEON64_CFLAGS) lib/arch/ssse3/codec.o: CFLAGS + = $(SSSE3_CFLAGS) lib/arch/sse41/codec.o: CFLAGS + = $(SSE41_CFLAGS) lib/arch/sse42/codec.o: CFLAGS + = $(SSE42_CFLAGS) lib/arch/avx/codec.o: CFLAGS + = $(AVX_CFLAGS) # so lib/arch/avx2/codec.so: CFLAGS + = $(AVX2_CFLAGS) lib/arch/neon32/codec.so: CFLAGS + = $(NEON32 _ CFLAGS) lib/arch/neon64/codec.so: CFLAGS + = $(NEON64_CFLAGS) lib/arch/ssse3/codec.so: CFLAGS + = $(SSSE3_CFLAGS) lib/arch/sse41/codec.so: CFLAGS + = $(SSE41_CFLAGS) lib/arch/sse42/codec.so: CFLAGS + = $(SSE42_CFLAGS) lib/arch/avx/codec.so: CFLAGS + = $(AVX_CFLAGS)% .o:% .c $(CC) $(CFLAGS)-o $@-c $<%. So:%. C $(CC) $(CFLAGS)-shared-fPIC-o $@-c $< analyze: clean scan-build-- use-analyzer= `which clang`-status-bugs make clean: rm-f bin/base64 bin/base64.o lib/libbase64.o lib/libbase64.so lib/config.h $(OBJS)

It doesn't matter if I don't understand. Makefile is so complicated and I can't understand it. I just modified it by feeling, and then it just happens to run. Note that the indentation of Makefile must use "\ t", otherwise an error will be reported if it does not conform to grammar.

Then we compile:

AVX2_CFLAGS=-mavx2 SSSE3_CFLAGS=-mssse3 SSE41_CFLAGS=-msse4.1 SSE42_CFLAGS=-msse4.2 AVX_CFLAGS=-mavx make lib/libbase64.so

This gives us the libbase64.so dynamic library (in lib). There are also various SIMD options turned on by the way. You can close it if you don't need it.

The magic reform begins.

Of course, this is just magic, not alchemy, so it takes effort. We need to bridge the dynamic library manually. First, we need to see what parameters are needed for the function we want to call. These two definitions are simple, and we need to pass in:

Const char * src size_t srclen char * out size_t * outlen int flagsvoid base64_encode (const char * src, size_t srclen, char * out, size_t * outlen, int flags); int base64_decode (const char * src, size_t srclen, char * out, size_t * outlen, int flags)

Then we can start writing ffi bridge programs. First of all, include all the required libraries, note that there is no harm in multi-use local, the use of local can effectively query from the local and avoid inefficient global queries. Even the functions in other packages can be local down to improve performance.

The words of dynamic library are referenced by special ffi.load.

Then define a _ M to wrap our library. This is very similar to JavaScript. JavaScript has window in the browser and Lua has _ G. We should try our best to avoid throwing the encapsulated library directly to the whole world, so it is a good way to encapsulate it.

Init local ffi = require "ffi" local floor = math.floor local ffi_new = ffi.new local ffi_str = ffi.string local ffi_typeof = ffi.typeof local C = ffi.C local libbase64 = ffi.load (". / libbase64.so")-- change this path when needed. Local _ M = {_ VERSION = '0.0.1'}

Then declare the ABI interface with ffi.cdef, which is even easier here, just copy the function declaration in the header file of the source code:

-- cdef ffi.cdef [[void base64_encode (const uint8_t * src, size_t srclen, uint8_t * out, size_t * outlen, size_t flags); int base64_decode (const uint8_t * src, size_t srclen, uint8_t * out, size_t * outlen, size_t flags);]

Then there is the most important type conversion:

-- define types local uint8t = ffi_typeof ("uint8_t [?]")-- uint8_t * local psizet = ffi_typeof ("size_t [1]")-- size_t *-- package function function _ M.base64_encode (src, flags) local dlen = floor ((# src * 8 + 4) / 6) local out = ffi_new (uint8t, dlen) local outlen = ffi_new (psizet, 1) libbase64.base64_encode (src, # src, out Outlen, flags) return ffi_str (out, outlen [0]) end function _ M.base64_decode (src, flags) local dlen = floor ((# src + 1) * 6 / 8) local out = ffi_new (uint8t, dlen) local outlen = ffi_new (psizet, 1) libbase64.base64_decode (src, # src, out, outlen, flags) return ffi_str (out, outlen [0]) end

We use ffi_typeof to define the data types to be mapped, and then use ffi_new to instantiate them and allocate memory space. Specifically:

We have defined two data types, where local uint8t = ffi_typeof ("uint8_t [?]") Type is used to transfer a string, followed by a question mark for the ffi_new function in local out = ffi_new (uint8t, dlen), whose second argument specifies the length when the data type is instantiated. This gives us an empty array of strings to hold the results returned by the C function. The dlen here calculates the length after the source string base64 encode, and allocates that length.

Similarly, local psizet = ffi_typeof ("size_t [1]") specifies a size_t * type. In C language, arrays are pointers, that is, size_t [0] is equivalent to site_t*. So we get a pointer to the size_t type by dividing the size_t array with only one element. Then when local outlen = ffi_new (psizet, 1), the following parameter also says 1, but it doesn't matter what is written here, it just doesn't support passing empty space, so we are equivalent to passing a placeholder.

When using this value, we also follow the pattern of the array: return ffi_str (out, outlen [0]).

It is important to note that require "ffi" and ffi.load must be placed at the bottom of the code, otherwise table overflow will occur.

Finally, the file looks like this:

-- [ffi-base64.lua @ version 20201228 author karminski 1 @ author karminski]-init local ffi = require "ffi" local floor = math.floor local ffi_new = ffi.new local ffi_str = ffi.string local ffi_typeof = ffi.typeof local C = ffi.C local libbase64 = ffi.load (". / libbase64.so")-- change this path when needed. Local _ M = {_ VERSION = '0.0.1'}-cdef ffi.cdef [[void base64_encode (const uint8_t * src, size_t srclen, uint8_t * out, size_t * outlen, size_t flags); int base64_decode (const uint8_t * src, size_t srclen, uint8_t * out, size_t * outlen, size_t flags) ]-- define types local uint8t = ffi_typeof ("uint8_t [?]")-- uint8_t * local psizet = ffi_typeof ("size_t [1]")-- size_t *-- package function function _ M.base64_encode (src, flags) local dlen = floor ((# src * 8 + 4) / 6) local out = ffi_new (uint8t, dlen) local outlen = ffi_new (psizet, 1) libbase64.base64_encode (src # src, out, outlen, flags) return ffi_str (out, outlen [0]) end function _ M.base64_decode (src, flags) local dlen = floor ((# src + 1) * 6 / 8) local out = ffi_new (uint8t, dlen) local outlen = ffi_new (psizet, 1) libbase64.base64_decode (src, # src, out, outlen, flags) return ffi_str (out, outlen [0]) end return _ M

All right, we're done. Let's write a demo and try it:

-- main.lua local ffi_base64 = require "ffi-base64" local target = "https://example.com" local r = ffi_base64.base64_encode (target, 0) print (" base64 encode result:\ n ".. r) local r = ffi_base64.base64_decode (r, 0) print (" base64 decode result:\ n ".. r) root@router02:/data/works/libbase64-ffi# luajit-v LuaJIT 2.1.0-beta3-- Copyright (C) 2005-2020 Mike Pall. Https://luajit.org/ root@router02:/data/works/libbase64-ffi# luajit. / main.lua base64 encode result: aHR0cHM6Ly9leGFtcGxlLmNvbQ== base64 decode result: https://example.com Thank you for your reading. The above is the content of "what is C language dynamic Library". After the study of this article, I believe you have a deeper understanding of what C language dynamic Library is, and the specific usage needs to be verified by practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.