Shulou(Shulou.com)06/01 Report--

This article shows you how to identify end-to-end CAPTCHA based on Serverless cloud function SCF+Kaggle from training to deployment. The content is concise and easy to understand, which will definitely brighten your eyes. I hope you can get something through the detailed introduction of this article.

With the upgrading of CAPTCHA technology, the traditional CAPTCHA recognition algorithm has become more and more useless. In recent years, artificial intelligence has developed rapidly, especially in the area of deep learning neural network, where a variety of algorithms and models emerge one after another.

Today, we will try to deploy an end-to-end universal CAPTCHA recognition model with the help of Kaggle+SCF rapid training. The real CAPTCHA recognition is an one-stop service from entry to application, ~

Effect display

Step 1: learn about kaggle

Students who have never been in a data science competition may not know much about kaggle.

Kaggle is the world's largest data science community with powerful tools and resources to help you achieve your data science goals.

This is the self-introduction of kaggle (official website). To put it simply, kaggle is the largest data science exchange community in the world, with many competitions and data sets about data science, and provides some environment and tools for online analysis of data science, which has attracted a large number of data science enthusiasts all over the world.

Here we mainly use the kernel environment in kaggle's Notebooks service to quickly train our CAPTCHA recognition model in the cloud.

You may ask if it is not possible to train locally, why do you have to go all the way to the cloud? Haha, this is really not a hassle. Ordinary people's computer computing power is actually limited, and the training model needs the support of strong GPU computing power, otherwise it will be trained until the year of the Monkey.

Let's take a look at the configuration of the kernel environment on kaggle:

CPU 4 core, 16 GB running memory

GPU 2 Core 13 GB running memory

Each kernel has a running time of 9 hours, and GPU resources are used for 30 hours per week. In addition to hardware resources, kernel environment has been configured with some common machine learning libraries, including Pytorch, Tensorflow 2, etc., its machine learning environment is out of the box, zero configuration, zero maintenance.

Kaggle Notebooks run in a remote computational environment. We provide the hardware-you need only worry about the code.

As stated in the official kaggle notebooks documentation, kaggle provides you with a free hardware and machine learning environment, and the only thing you need to care about is your code. The key to such a good thing is provided free of charge, so just choose it decisively to train the model.

Step 2: register a kaggle account and create a new kernel environment

Account registration, new kernel and other related issues, there are many related articles online, I will not go into details here.

Step 3: clone git warehouse, modify to your own CAPTCHA data set

Here, based on the github.com/nickliqian/cnn_captcha project, I upgraded the original project to Tensorflow 2.0, and then made a general CAPTCHA identification scheme of kaggle training + SCF deployment.

Now all you have to do is clone my modified warehouse https://gitee.com/LoveWJG/tflite_train locally.

Then configure the training parameters according to the readme in the project and replace your own CAPTCHA data set.

Step 4: upload the project to kaggle to start training

Then the configured project is compressed and uploaded to kaggle and directly decompressed and trained in accordance with the instructions.

20000 CAPTCHA codes are used here, and about 10000 rounds of training, which takes about 30 minutes, is quite powerful. After training, you can download the model and log files locally according to the readme file in the warehouse, and then convert the model to tflite format locally (easy to use on mobile, local identification code). If the model file is too large, you can also run the tflite.py program locally to quantify the tflite model, which can probably reduce the model file to the original 1ax 4. Eventually you should get a model file in .tflite format.

Step 5: quickly deploy the CAPTCHA identification model using cloud function

For the creation, configuration and release of cloud functions, please refer to my previous series of articles, so I won't go into details here.

Create a new python blank cloud function, and then fill in the code in the scf.py file and save it in index.py.

#-*-coding:utf-8-*-import ioimport jsonimport osimport timeimport numpy as npimport tensorflow as tffrom PIL import Imagemodel_path = "model_quantized.tflite" # Model file address chars = '23456789abcdefghjkmpqrstuvwxy' # CAPTCHA character The order should be the same as in config.json # Load TFLite model and allocate tensors.interpreter = tf.lite.Interpreter (model_path=model_path) interpreter.allocate_tensors () # Get input and output tensors.input_details = interpreter.get_input_details () output_details = interpreter.get_output_details () # convert the CAPTCHA data into the model input format def img2input (img, width Height): tmpe_array = [serverless] for i in range (height): for j in range (width): pixel = img.getpixel ((j, I)) tmpe_array.append ((0.3*pixel [0] + 0.6*pixel [1] + 0.1*pixel [2]) / 255) tmpe_array = np.array (tmpe_array) .astype ('float32') input_array = np.expand_dims (tmpe_array) Axis=0) return input_array# authentication code def predict (image): captcha_image = Image.open (io.BytesIO (image)) image_np_expanded = img2input (captcha_image, 100,50) interpreter.set_tensor (input_details [0] ['index'] Image_np_expanded) interpreter.invoke () output_data = interpreter.get_tensor (output_details [0] ['index']) codes =' 'for i in output_data [0]: codes + = chars [I] return codes# api Gateway response Integration def apiReply (reply, txt=False, content_type='application/json', code=200): return {"isBase64Encoded": False, "statusCode": code "headers": {'Content-Type': content_type}, "body": json.dumps (reply, ensure_ascii=False) if not txt else str (reply)} # Cloud function entry def main_handler (event, context): return apiReply ({"ok": False if not' image' in event.keys () else True "message": "invalid request parameter" if not 'image' in event.keys () else predict (event [' queryString'] ['image'])})

Upload the model file to the root directory of the cloud function, and then configure your own CAPTCHA to identify the model parameters

Model_path = "model_quantized.tflite" # Model file address chars = '23456789abcdefghjkmpqrstuvwxy' # CAPTCHA characters in the same order as in config.json

After that, add an API gateway trigger to our cloud function, enable response integration, and then publish and launch.

If there is no problem, you only need to GET and you can return the CAPTCHA recognition result.

Api gateway +? base64Image=base64 encoded CAPTCHA data

The editor takes you to train and deploy a general CAPTCHA recognition model from scratch. Once again, we see how convenient and fast it is in the process of developing online applications based on Serverless.

The above is how to identify end-to-end CAPTCHA based on Serverless cloud function SCF+Kaggle. Have you learned any knowledge or skills from training to deployment? If you want to learn more skills or enrich your knowledge reserve, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.