Shulou(Shulou.com)06/01 Report--

Most people do not understand the knowledge points of this article "how to use webcoekt to play JPEG picture stream in the front end", so the editor summarizes the following content, detailed content, clear steps, and has a certain reference value. I hope you can get something after reading this article. Let's take a look at this "how to use webcoekt to play JPEG picture stream in the front end" article.

I. brief introduction

Since webcoekt is connected based on tcp, in theory, all browsers can handle binaries with proprietary protocols. If we need to play video, we can decode the video data at the back end and push the picture directly to the front end of webcoekt. Then the front end can receive the picture through websocket and display it in img or canvas. Of course, this is my own idea, and it should be possible to do it. The following technical support is required to do the following:

Direct ffmpeg transcoding to jpeg image stream at backend

The backend custom playback protocol includes basic instructions such as play, stop, pause, faster, and slower.

The backend needs to provide websocket support to send pictures to the front end.

The front end needs to accept the picture stream and display it.

Back-end ffmpeg decoding is not explained here, I have a lot of libraries, need to contact me separately to buy, the front-end display jpg stream, here with the help of the front-end display picture put the practice, use picture base64 data! The front-end HTML displays binary jpeg images: picture stream = > base64 encoding of binary-to-image = > set to the src of img, such as the front-end code:

Binary to base64 via arraybuffer:

Function arrayBufferToBase64 (buffer) {var binary =''; var bytes = new Uint8Array (buffer); var len = bytes.byteLength; for (var I = 0; I

< len; i++) { binary += String.fromCharCode(bytes[i]); } return window.btoa(binary); } 最后显示： var player = document.getElementById('player'); var url= arrayBufferToBase64(data); player.src=_'data:image/jpeg;base64,'+url; 当然，还有其他的方式： var wsCtrl = new WebSocket("ws://127.0.0.1/ctrl/");//Establish channel code....var wsVideo = new WebSocket("ws://127.0.0.1/video/channel1");wsVideo.onmessage = function(evt){ //Method 1 document.getElementById("img1").src = URL.createObjectURL(evt.data); //Method 2 var read = new FileReader(); read.onload = function(e) { document.getElementById("img2").src = e.target.result; } read.readAsDataURL(evt.data);}ws.onmessage = function(evt) { if(typeof(evt.data)=="string"){ //textHandler(JSON.parse(evt.data)); }else{ var reader = new FileReader(); reader.onload = function(evt){ if(evt.target.readyState == FileReader.DONE){ var url = evt.target.result; alert(url); var img = document.getElementById("imgDiv"); img[xss_clean] = "

Reader.readAsDataURL (evt.data);}}

Websocket Open Source Project on C++: websocketpp, QWebSocketServer

2. Websocket plays the picture stream

To reduce the complexity, I use java's websocket to send the picture stream (of course, C++ 's library I also used in a practical project called WebSocket encapsulation dll project, need to buy the source code privately), the front end uses an img tag to display the picture, here is a description Simulate sending pictures in the background (here are only pictures, not streams. If the streams are sent directly and continuously, you can need ffmpeg transcoding)

First, the front-end code is as follows:

Websocket displays binary image stream $(document) .ready (function () {$("# send") .click (function () {/ var content = $("# content") .val () $.ajax ({url: "/ test/send", data: {/ / content: content content: ""}, success: function (result) {console.log ("request successful!");}}) });}); WebSocket play pictures

Request picture var websocket = {send: function (str) {}}; _ window.onload = function () {if (! 'WebSocket' in window) return; webSocketInit ();} Function webSocketInit () {/ / Connect to the server endpoint websocket = new WebSocket ("ws://127.0.0.1:8080/image/show"); / / successfully establish a connection websocket.onopen = function () {console.log ("successfully connect to the server") Websocket.send ("successfully connected to the server");} / / received message websocket.onmessage = function (event) {/ / text packet if (typeof (event.data) = = "string") {/ / JSON.parse (evt.data) console.log ("received message from server:" + event.data) / / Picture packet Blob} else {var reader = new FileReader (); reader.onload = function (evt) {if (evt.target.readyState = = FileReader.DONE) {/ / base64 data var url = evt.target.result Document.getElementById ("player"). Src = url;}} reader.readAsDataURL (event.data);} / / connection error websocket.onerror = function () {console.log ("WebSocket connection error");}; / / connection closed websocket.onclose = function () {console.log ("WebSocket connection closed");} / / listen for the window close event. When the window closes, actively close the websocket connection _ window.onbeforeunload = function () {websocket.close ()};}

Each time you click send, you request a picture from the backend, and the backend sends out the changed image (I simply use websocket to send a group message, and you can use the variable parameters of websocket to associate websocket with http. This should be easy for me to discuss in a group that I don't know any more about here.)

Package com.easystudy.controller;import java.io.File;import java.io.FileInputStream;import java.io.IOException;import java.io.InputStream;import java.net.URL;import java.util.Random;import org.springframework.web.bind.annotation.GetMapping;import org.springframework.web.bind.annotation.RequestMapping;import org.springframework.web.bind.annotation.RequestParam;import org.springframework.web.bind.annotation.RestController;import com.easystudy.websocket.ImgEndPoint / * * @ File name: TestController.java * @ function description: image stream request request sending API (websocket sends pictures to web) * @ copyright Information: www.easystudy.com * @ Technical Exchange: 961179337 (QQ Group) * @ author: lixx2048@163.com * @ contact: 941415509 (QQ) * @ Development date: September 21, 2020 * @ Historical version: V1.0 * @ remarks: * / @ RestController@RequestMapping ("/ test") public class TestController {/ * * @ function description: send request API * @ copyright Information: www.easystudy.com * @ author: lixx2048@163.com * @ Development date: September 21, 2020 * @ remarks letter Information: * / @ SuppressWarnings ("unused") @ GetMapping ("/ send") public String reponseMsgToClient (@ RequestParam (name= "content) Required = true) String content) throws Exception {System.out.println ("start sending picture data") / / randomly select a picture and send int index = new Random (). NextInt (4) + 1; String imgName = index + ".jpg"; / / determine whether the picture exists URL url = getClass (). GetClassLoader (). GetResource (imgName); File file = new File (url.getFile ()) If (! file.exists ()) {return "picture not found!";} / / create input picture stream InputStream in = new FileInputStream (file); if (null = = in) {return "failed to open file!" } / / read picture data int size = (int) file.length (); byte [] buffer = new byte [size]; int count = in.read (buffer, 0, size); System.out.println ("file length:" + size + ", read length:" + count)) / / send picture data (theoretically, only peer-to-peer connections should be sent) ImgEndPoint.fanoutMessage (buffer); / / close the file stream try {in.close ();} catch (IOException e) {e.printStackTrace () } / / API response return "message [" + content+ "] was sent successfully!";}}

I have one more word here. If it is the video playback of audio and video applications, you can use websocket to transmit the picture stream, and then realize the custom player function by calculating and sending the fixed-point picture stream data to the front end. (Haikang Fluorite Cloud uses websocket to play the video stream, which is similar to that used by Haikang fluorite cloud.)

The above is about the content of this article on "how to use webcoekt to play JPEG picture stream in the front end". I believe we all have a certain understanding. I hope the content shared by the editor will be helpful to you. If you want to know more about the relevant knowledge, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.