Shulou(Shulou.com)06/02 Report--

Knowledge preparation

AVPacket: store compressed data (video corresponds to H.264 bitstream data, audio corresponds to AAC/MP3 bitstream data). To put it simply, it carries a NAL video unit or multiple NAL audio units. AVPacket holds the pre-decoding data of an NAL unit. The structure itself does not directly contain data, but has a pointer to the data field. The data in the AVPacket structure data passed to the avcodec_send_packet function begins with 00 00 00 01, indicating that it is in NALU format.

Member analysis of important structures

AVBufferRef * buf; / / the reference count of compressed data in the current AVPacket and the pointer address where the compressed data is saved (the space applied for for compressed data is here)

Uint8_t * data;// pointer address where compressed data is saved (data also points to data in buf)

Length of int size;// compressed data

Int stream_index;// video or audio index

Actual combat (building an AVPacket that contains a NAL unit (length nLen))

AVPacket pkt1, * packet = & pkt1

Av_new_packet (packet, nLen)

Memcpy (packet- > data, data, nLen)

Packet- > size = nLen

Packet- > stream_index = 0

Then you can add packet to the linked list and wait for a frame of data to be decoded, or call avcodec_decode_video2 to decode it. After decoding, you can call av_free_packet or av_packet_unref to release resources.

Dispel doubts

1) Why not request memory directly from packet- > data, and then copy the data? As defined in the structure, AVBufferRef is only the reference count of the data. It can be NULL, which means that there are no references.

So the above code modification (not recommended):

Av_init_packet (packet); / / initialize structures, especially AVBufferRef * buf, to avoid

Packet- > data = (uint8_t *) malloc (sizeof (uint8_t) * nByte)

Memcpy (packet- > data, data, nLen)

Packet- > size = nLen

Packet- > stream_index = 0

Note: av_init_packet (packet); / / initialize structures, especially AVBufferRef * buf, to avoid accessing illegal pointer addresses during decoding.

Not recommended: av_free_packet or av_packet_unref cannot be used for resource release. The memory requested by packet- > data must be released manually, because these two functions release resources for buf in the AVPacket structure, not data.

Void av_free_packet (AVPacket * pkt)

{

If (pkt) {

If (pkt- > buf)

Av_buffer_unref (& pkt- > buf)

Pkt- > data = NULL

Pkt- > size = 0

Av_packet_free_side_data (pkt)

}

}

Void av_packet_unref (AVPacket * pkt)

{

Av_packet_free_side_data (pkt)

Av_buffer_unref (& pkt- > buf)

Av_init_packet (pkt)

Pkt- > data = NULL

Pkt- > size = 0

}

2) Why is the data in the AVBufferRef the same as the outer data pointer, but the size length is not the same

According to

Int av_new_packet (AVPacket * pkt, int size)

{

AVBufferRef * buf = NULL

Int ret = packet_alloc (& buf, size)

If (ret

< 0) return ret; av_init_packet(pkt); pkt->

Buf = buf

Pkt- > data = buf- > data

Pkt- > size = size

Return 0

}

You can see that the two data pointers point to the same block of memory, but when the memory is requested, the bytes are aligned

Apply for more AV_INPUT_BUFFER_PADDING_SIZE bytes of data as the filling at the end

Void av_init_packet (AVPacket * pkt)

{

Pkt- > pts = AV_NOPTS_VALUE

Pkt- > dts = AV_NOPTS_VALUE

Pkt- > pos =-1

Pkt- > duration = 0

# if FF_API_CONVERGENCE_DURATION

FF_DISABLE_DEPRECATION_WARNINGS

Pkt- > convergence_duration = 0

FF_ENABLE_DEPRECATION_WARNINGS

# endif

Pkt- > flags = 0

Pkt- > stream_index = 0

Pkt- > buf = NULL

Pkt- > side_data = NULL

Pkt- > side_data_elems = 0

}

/ / AV_INPUT_BUFFER_PADDING_SIZE is for data alignment and convenient data access

Static int packet_alloc (AVBufferRef * * buf, int size)

{

Int ret

If (size

< 0 || size >

= INT_MAX-AV_INPUT_BUFFER_PADDING_SIZE)

Return AVERROR (EINVAL)

Ret = av_buffer_realloc (buf, size + AV_INPUT_BUFFER_PADDING_SIZE)

If (ret

< 0) return ret; memset((*buf)->

Data + size, 0, AV_INPUT_BUFFER_PADDING_SIZE)

Return 0

}

Create an AVBufferRef instance and apply for size-length memory allocation to the data in the AVBufferRef instance

Int av_buffer_realloc (AVBufferRef * * pbuf, int size)

{

AVBufferRef * buf = * pbuf

Uint8_t * tmp

If (! buf) {

/ * allocate a new buffer with av_realloc (), so it will be reallocatable

* later * /

Uint8_t * data = av_realloc (NULL, size)

If (! data)

Return AVERROR (ENOMEM)

Buf = av_buffer_create (data, size, av_buffer_default_free, NULL, 0)

If (! buf) {

Av_freep & data)

Return AVERROR (ENOMEM)

}

Buf- > buffer- > flags | = BUFFER_FLAG_REALLOCATABLE

* pbuf = buf

Return 0

} else if (buf- > size = = size)

Return 0

If (! (buf- > buffer- > flags & BUFFER_FLAG_REALLOCATABLE) | |

! av_buffer_is_writable (buf) | | buf- > data! = buf- > buffer- > data) {

/ * cannot realloc, allocate a new reallocable buffer and copy data * /

AVBufferRef * new = NULL

Av_buffer_realloc (& new, size)

If (! new)

Return AVERROR (ENOMEM)

Memcpy (new- > data, buf- > data, FFMIN (size, buf- > size))

Buffer_replace (pbuf, & new)

Return 0

}

Tmp = av_realloc (buf- > buffer- > data, size)

If (! tmp)

Return AVERROR (ENOMEM)

Buf- > buffer- > data = buf- > data = tmp

Buf- > buffer- > size = buf- > size = size

Return 0

}

/ / release AVBufferRef application memory

Void av_buffer_unref (AVBufferRef * * buf)

{

If (! buf | |! * buf)

Return

Buffer_replace (buf, NULL)

}

Static void buffer_replace (AVBufferRef * dst, AVBufferRef * * src)

{

AVBuffer * b

B = (* dst)-> buffer

If (src) {

* * dst = * * src

Av_freep (src)

} else

Av_freep (dst)

If (atomic_fetch_add_explicit (& b-> refcount,-1, memory_order_acq_rel) = = 1) {

B-> free (b-> opaque, b-> data)

Av_freep & b)

}

}

3)

The data are as follows:

00 00 00 01 61 e1 40 01 58 2b fb 22 ff 29 7b 3f 6f 67 2f 29 fa 25 53 68 78 46 b1

The print error when calling the avcodec_send_packet function is as follows:

2018-01-06 15:06:05 ms:887:nal_unit_type: 1, nal_ref_idc: 3

2018-01-06 15:06:05 ms:888:non-existing PPS 0 referenced

2018-01-06 15:06:05 ms:888:decode_slice_header error

2018-01-06 15:06:05 ms:888:no frame!

When the data is as follows, an image can be parsed correctly.

00 00 00 01 67 42 00 2a 96 35 40 f0 04 4f cb 37 01 01 01 40 00 01 c2 00 00 57 e4

01 00 00 00 01 68 ce 3c 80 00 00 00 01 06 e5 01 ef 80 00 00 03 00 00 00 01 65 b8

00 00 52 58 00 00 27 f5 d4 48 7e b4 41 07 24 60 95 2c 92 37 68 75 63 4c ad 3f b1

Obviously, 67 is SPS,68, and then 65 is the Keyframe, and then the image starts to come out.

Structure definition

/ * *

* A reference to a data buffer.

*

* The size of this struct is not a part of the public ABI and it is not meant

* to be allocated directly.

, /

Typedef struct AVBufferRef {

AVBuffer * buffer

/ * *

* The data buffer. It is considered writable if and only if

* this is the only reference to the buffer, in which case

* av_buffer_is_writable () returns 1.

, /

Uint8_t * data

/ * *

* Size of data in bytes.

, /

Int size

} AVBufferRef

/ * *

* This structure stores compressed data. It is typically exported by demuxers

* and then passed as input to decoders, or received as output from encoders and

* then passed to muxers.

*

* For video, it should typically contain one compressed frame. For audio it may

* contain several compressed frames.

*

* AVPacket is one of the few structs in FFmpeg, whose size is a part of public

* ABI. Thus it may be allocated on stack and no new fields can be added to it

* without libavcodec and libavformat major bump.

*

* The semantics of data ownership depends on the buf or destruct (deprecated)

* fields. If either is set, the packet data is dynamically allocated and is

* valid indefinitely until av_free_packet () is called (which in turn calls

* av_buffer_unref () / the destruct callback to free the data). If neither is set

* the packet data is typically backed by some static buffer somewhere and is

* only valid for a limited time (e.g. Until the next read call when demuxing).

*

* The side data is always allocated with av_malloc () and is freed in

* av_free_packet ().

, /

Typedef struct AVPacket {

/ * *

* A reference to the reference-counted buffer where the packet data is

* stored.

* May be NULL, then the packet data is not reference-counted.

, /

AVBufferRef * buf

/ * *

* Presentation timestamp in AVStream- > time_base units; the time at which

* the decompressed packet will be presented to the user.

* Can be AV_NOPTS_VALUE if it is not stored in the file.

* pts MUST be larger or equal to dts as presentation cannot happen before

* decompression, unless one wants to view hex dumps. Some formats misuse

* the terms dts and pts/cts to mean something different. Such timestamps

* must be converted to true pts/dts before they are stored in AVPacket.

, /

Int64_t pts

/ * *

* Decompression timestamp in AVStream- > time_base units; the time at which

* the packet is decompressed.

* Can be AV_NOPTS_VALUE if it is not stored in the file.

, /

Int64_t dts

Uint8_t * data

Int size

Int stream_index

/ * *

* A combination of AV_PKT_FLAG values

, /

Int flags

/ * *

* Additional packet data that can be provided by the container.

* Packet can contain several types of side information.

, /

Struct {

Uint8_t * data

Int size

Enum AVPacketSideDataType type

} * side_data

Int side_data_elems

/ * *

* Duration of this packet in AVStream- > time_base units, 0 if unknown.

* Equals next_pts-this_pts in presentation order.

, /

Int duration

# if FF_API_DESTRUCT_PACKET

Attribute_deprecated

Void (* destruct) (struct AVPacket *)

Attribute_deprecated

Void * priv

# endif

Int64_t pos; /

< byte position in stream, -1 if unknown /** * Time difference in AVStream->

Time_base units from the pts of this

* packet to the point at which the output from the decoder has converged

* independent from the availability of previous frames. That is, the

* frames are virtually identical no matter if decoding started from

* the very first frame or from this keyframe.

* Is AV_NOPTS_VALUE if unknown.

* This field is not the display duration of the current packet.

* This field has no meaning if the packet does not have AV_PKT_FLAG_KEY

* set.

*

* The purpose of this field is to allow seeking in streams that have no

* keyframes in the conventional sense. It corresponds to the

* recovery point SEI in H.264 and match_time_delta in NUT. It is also

* essential for some types of subtitle streams to ensure that all

* subtitles are correctly displayed after seeking.

, /

Int64_t convergence_duration

} AVPacket

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.