Shulou(Shulou.com)05/31 Report--

This article mainly explains "Go language how to develop browser video stream rtsp to webrtc playback", interested friends may wish to have a look. The method introduced in this paper is simple, fast and practical. Next let the editor to take you to learn "Go language how to develop browser video stream rtsp to webrtc playback"!

1. Preface

Earlier, we tested the rtsp-to-hls mode and found that the delay is relatively large, which is not suitable for our needs. Next, let's try webrtc to see how it works.

Taking all things into consideration, we'd better find a front-end and front-end solution with go as the backend and better front-end compatibility to deal with webrtc, so that we can combine our previous cgo+onvif+gSoap implementation to obtain rtsp streams, and we can make more extensions according to the functional interfaces such as ptz and preset points that have been implemented.

2. Rtsp to webRTC

The following is a suitable open source solution found. The front end uses jQuery, bootstrap, etc., and the back end uses go+gin to implement and convert rtsp stream parsing to webRTC protocol to provide http-related interfaces to the front end, and configure rtsp address and stun address through config.json:

Click download

In addition, it also comes with stun, and you can configure your own stun address to facilitate private network penetration.

The initial test showed almost no delay, in line with expectations, and the use of jQuery+bootstrap+go+gin to do web is also in line with the use of our project.

3. Preliminary test results

The results are as follows:

4. Make changes in combination with our previous onvif+gSoap+cgo scheme

On the basis of this project, combined with the scheme of onvif+cgo+gSoap we studied earlier, we provide the interface to the web side of the relevant data obtained by onvif, adding ptz, focusing, zooming and other functions.

We add a new post interface to http.go: HTTPAPIServerStreamPtz for ptz and HTTPAPIServerStreamPreset for preset point related operations.

The following is part of the code, did not do too much optimization, but only achieved ptz, focusing and scaling, is considered to open the way, the specific project needs can be optimized.

4.1 go backend modification

A new interface has been added and the previous cgo+onvif+gSoap content has been integrated into the project. No more optimization has been made in the project as a whole, just to demonstrate and provide an idea:

Http.go (two post interfaces, ptz and preset, are added and handled in cgo mode):

Package main/*#cgo CFLAGS:-I. /-I / usr/local/#cgo LDFLAGS:-L. / build-lc_onvif_static-lpthread-ldl-lssl-lcrypto#include "client.h" # include "malloc.h" * / import "C" import ("encoding/json", "fmt", "log", "net/http", "os", "sort", "strconv", "time", "unsafe"github" .com / deepch/vdk/av "webrtc" github.com/deepch/vdk/format/webrtcv3 "github.com/gin-gonic/gin") type JCodec struct {Type string} func serveHTTP () {gin.SetMode (gin.ReleaseMode) router: = gin.Default () router.Use (CORSMiddleware ()) if _ Err: = os.Stat (". / web") ! os.IsNotExist (err) {router.LoadHTMLGlob ("web/templates/*") router.GET ("/", HTTPAPIServerIndex) router.GET ("/ stream/player/:uuid", HTTPAPIServerStreamPlayer)} router.POST ("/ stream/receiver/:uuid", HTTPAPIServerStreamWebRTC) / / add a new post interface router.POST ("/ stream/ptz/", HTTPAPIServerStreamPtz) router.POST ("/ stream/preset/") HTTPAPIServerStreamPreset) router.GET ("/ stream/codec/:uuid", HTTPAPIServerStreamCodec) router.POST ("/ stream", HTTPAPIServerStreamWebRTC2) router.StaticFS ("/ static", http.Dir ("web/static")) err: = router.Run (Config.Server.HTTPPort) if err! = nil {log.Fatalln ("Start HTTP Server error", err)}} / HTTPAPIServerIndex indexfunc HTTPAPIServerIndex (c * gin.Context) {_ All: = Config.list () if len (all) > 0 {c.Header ("Cache-Control", "no-cache, max-age=0, must-revalidate, no-store") c.Header ("Access-Control-Allow-Origin", "*") c.Redirect (http.StatusMovedPermanently, "stream/player/" + all [0])} else {c.HTML (http.StatusOK, "index.tmpl") Gin.H {"port": Config.Server.HTTPPort, "version": time.Now (). String ()}} / / HTTPAPIServerStreamPlayer stream playerfunc HTTPAPIServerStreamPlayer (c * gin.Context) {_, all: = Config.list () sort.Strings (all) c.HTML (http.StatusOK, "player.tmpl", gin.H {"port": Config.Server.HTTPPort Suuid: c.Param ("uuid"), "suuidMap": all, "version": time.Now () .String () })} / / HTTPAPIServerStreamCodec stream codecfunc HTTPAPIServerStreamCodec (c * gin.Context) {if Config.ext (c.Param ("uuid")) {Config.RunIFNotRun (c.Param ("uuid")) codecs: = Config.coGe (c.Param ("uuid")) if codecs = = nil {return} var tmpCodec [] JCodec for _ Codec: = range codecs {if codec.Type ()! = av.H264 & & codec.Type ()! = av.PCM_ALAW & & codec.Type ()! = av.PCM_MULAW & & codec.Type ()! = av.OPUS {log.Println ("Codec Not Supported WebRTC ignore this track" Codec.Type () continue} if codec.Type (). IsVideo () {tmpCodec = append (tmpCodec, JCodec {Type: "video"})} else {tmpCodec = append (tmpCodec, JCodec {Type: "audio"})} b Err: = json.Marshal (tmpCodec) if err = = nil {_, err = c.Writer.Write (b) if err! = nil {log.Println ("Write Codec Info error" Err) return}} / / HTTPAPIServerStreamWebRTC stream video over WebRTCfunc HTTPAPIServerStreamWebRTC (c * gin.Context) {if! Config.ext (c.PostForm ("suuid")) {log.Println ("Stream Not Found") return} Config.RunIFNotRun (c.PostForm ("suuid")) codecs: = Config.coGe (c.PostForm ("suuid")) if codecs = = nil {log.Println ("Stream Codec Not Found") return} var AudioOnly bool if len (codecs) = = 1 & & codecs [0] .Type (). IsAudio () {AudioOnly = true } muxerWebRTC: = webrtc.NewMuxer (webrtc.Options {ICEServers: Config.GetICEServers () ICEUsername: Config.GetICEUsername (), ICECredential: Config.GetICECredential (), PortMin: Config.GetWebRTCPortMin (), PortMax: Config.GetWebRTCPortMax ()} answer, err: = muxerWebRTC.WriteHeader (codecs, c.PostForm ("data")) if err! = nil {log.Println ("WriteHeader", err) return} _ Err = c.Writer.Write ([] byte (answer)) if err! = nil {log.Println ("Write", err) return} go func () {cid, ch: = Config.clAd (c.PostForm ("suuid")) defer Config.clDe (c.PostForm ("suuid") Cid) defer muxerWebRTC.Close () var videoStart bool noVideo: = time.NewTimer (10 * time.Second) for {select {case Zoom + focus + Stop focusing

App.js:

Let stream = new MediaStream (); let suuid = $('# suuid'). Val (); let config = {iceServers: [{urls: ["stun:stun.l.google.com:19302"]}}; const pc = new RTCPeerConnection (config); pc.onnegotiationneeded = handleNegotiationNeededEvent;let log = msg = > {document.getElementById ('div') [xss_clean] + = msg +'

'} pc.ontrack = function (event) {stream.addTrack (event.track); videoElem.srcObject = stream; log (event.streams.length +' track is delivered')} pc.oniceconnectionstatechange = e = > log (pc.iceConnectionState) async function handleNegotiationNeededEvent () {let offer = await pc.createOffer (); await pc.setLocalDescription (offer); getRemoteSdp ();} $(document) .ready (function () {$('#'+ suuid) .addClass ('active'); getCodecInfo ();}) Function getCodecInfo () {$.get (".. / codec/" + suuid, function (data) {try {data = JSON.parse (data);} catch (e) {console.log (e);} finally {$.each (data,function (index,value) {pc.addTransceiver (value.Type, {'direction':' sendrecv'})}})) } let sendChannel = null;function getRemoteSdp () {$.post (".. / receiver/" + suuid, {suuid: suuid, data: btoa (pc.localDescription.sdp)}, function (data) {try {pc.setRemoteDescription (new RTCSessionDescription ({type: 'answer', sdp: atob (data)})} catch (e) {console.warn (e);}})) } function ptz (direction) {$.post (".. / ptz/", direction, function (data, status) {console.debug (_ "Data:" + data + "nStatus:" + status);} function funTopClick () {console.debug ("top click"); ptz ("action=1")} function funDownClick () {console.debug ("down click"); ptz ("action=2")} function funLeftClick () {console.debug ("left click") Ptz ("action=3")} function funRightClick () {console.debug ("right click"); ptz ("action=4")} function funStopClick () {console.debug ("stop click"); ptz ("action=9")} function funZoomClick (direction) {console.debug ("zoom click" + direction); ptz ("action=" + direction)} function funFocusClick (direction) {console.debug ("focus click" + direction); ptz ("action=" + direction)}

A fan button and two groups of buttons are mainly added, and then the click of the button is combined into app.js for processing, and app.js sends a post request to call the go back-end interface.

4.3 Project structure and compilation and running

The structure of the project is as follows, and some files have been backed up, which can not be used actually:

$tree-a-I ".github | .idea | build". ├── .gitignore ├── CMakeLists.txt ├── Dockerfile ├── LICENSE ├── README.md ├── build.cmd ├── client.c ├── client.h ├── config.go ├── config.json ├── config.json.bak ├── doc │ ├── demo2.png │ ├── demo3.png │ └── demo4.png ├── go.mod go.sum ── http.go ├── main.go ├── main.go.bak ├── renovate.json ├── soap │ ├── DeviceBinding.nsmap │ ├── ImagingBinding.nsmap │ ├── MediaBinding.nsmap │ ├── PTZBinding.nsmap │ ├── PullPointSubscriptionBinding.nsmap │ ├── RemoteDiscoveryBinding.nsmap │ ├── custom README.txt chrono_duration.cpp ├── chrono_duration.h │ │ ├── chrono_time_point.cpp │ │ ├── chrono_time_point.h │ │ ├── duration.c │ │ ├── duration.h │ │ ├── float128.c │ │ ├── float128.h int128.c int128.h ├── long_double.c │ │ ├── long_double.h │ │ ├── long_time.c │ │ ├── long_time.h │ │ ├── qbytearray_base64.cpp │ │ ├── qbytearray_base64.h │ │ ├── qbytearray_hex.cpp qbytearray_hex.h qdate.cpp │ ├── qdate.h │ │ ├── qdatetime.cpp │ │ ├── qdatetime.h │ │ ├── qstring.cpp │ │ ├── qstring.h │ │ ├── qtime.cpp │ │ qtime.h struct_timeval.c struct_timeval.h struct _ tm.c │ │ ├── struct_tm.h │ │ ├── struct_tm_date.c │ │ └── struct_tm_date.h │ ├── dom.c │ ├── dom.h │ ├── duration.c │ ├── duration.h │ ├── mecevp.c mecevp.h onvif.h smdevp .c │ ├── smdevp.h │ ├── soapC.c │ ├── soapClient.c │ ├── soapH.h │ ├── soapStub.h │ ├── stdsoap2.h │ ├── stdsoap2_ssl.c │ ├── struct_timeval.c │ ├── struct_timeval.h │ threads.c threads.h typemap.dat ── wsaapi.c │ ├── wsaapi.h │ ├── wsdd.nsmap │ ├── wsseapi.c │ └── wsseapi.h ├── stream.go └── web ├── static │ ├── css │ │ ├── bootstrap-grid.css │ │ bootstrap-grid.css.map bootstrap-grid.min.css │ │ ├── bootstrap-grid.min.css.map │ │ ├── bootstrap-reboot.css │ │ ├── bootstrap-reboot.css.map │ │ ├── bootstrap-reboot.min.css │ │ ├── bootstrap-reboot.min.css.map │ │ ├── bootstrap.css │ bootstrap.css.map │ │ ├── bootstrap.min.css │ │ ├── bootstrap.min.css.map │ │ └── shanxing.css │ └── js │ ├── adapter-latest.js │ ├── app.js │ ├── bootstrap.bundle.js │ ├── bootstrap.bundle.js.map ├── bootstrap.bundle.min.js │ ├── bootstrap.bundle.min.js.map │ ├── bootstrap.js │ ├── bootstrap.js.map │ ├── bootstrap.min.js │ ├── bootstrap.min.js.map │ └── jquery-3.4.1.min.js └── templates ├── index.tmpl └── player.tmpl8 directories 111 files

I won't say much about cgo, onvif and gSoap here. If it's not clear, you can see the previous summary. Gin, bootstramp, jQuery also need some learning and reserve of front and rear concepts, and they are also sporadically distributed in other classified summaries. If it's not clear, you can take a look, so we won't say any more here.

Compile and run:

GOOS=linux GOARCH=amd64 CGO_ENABLE=1 GO111MODULE=on go run * .go

Remember to modify the dependence on the go version in go.mod. According to the cgo problem, at least 1.18 or more is required at present, otherwise there may be a segmentation violation when running ptz.

Module github.com/deepch/RTSPtoWebRTCgo 1.18require (github.com/deepch/vdk v0.0.0-20220309163430-c6529706436c github.com/gin-gonic/gin v1.7.7) 4.4 result display

Interface effect:

Dynamically debug ptz:

Dynamic debug scaling:

Dynamic debugging and focusing:

At this point, I believe you have a deeper understanding of "Go language how to develop browser video stream rtsp to webrtc playback". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.