Shulou(Shulou.com)05/31 Report--

This article mainly talks about "what is the method of Android camera development". Interested friends may wish to have a look at it. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn "what is the method of Android camera development?"

I. two ways to develop camera applications in Android

The Android system provides two ways to use mobile camera resources to achieve shooting functions. one is to call the system camera component directly through Intent. This method is fast and convenient, and is suitable for scenes where photos are obtained directly, such as uploading photo albums, Weibo, sending photos on moments, and so on. The other is to use camera API to customize the camera, this method is suitable for scenes that need to customize the camera interface or develop special camera functions, such as clipping photos, filtering, adding stickers, expressions, location tags, and so on. This article focuses on how to use camera API to customize custom cameras.

two。 Analysis of key classes in camera API

The implementation of shooting function through camera API involves the following key classes and interfaces:

Camera: the primary class for managing and manipulating camera resources. It provides a complete camera underlying interface, supporting camera resource switching, setting preview / shooting size, setting aperture, exposure, focus and other related parameters, obtaining preview / shooting frame data and other functions. The main methods are as follows:

Open (): get the camera instance.

SetPreviewDisplay (SurfaceHolder): binds the surface that draws the preview image. Surface is a handle to the original image buffer (raw buffer) of the screen window, through which you can get the corresponding canvas on this screen, and then complete the work of drawing View on the screen. Camera and surface can be connected through surfaceHolder, and when camera and surface are connected, the preview frame data obtained by camera can be displayed on the screen through surface.

SetPrameters sets camera parameters, including front and rear cameras, flash mode, focus mode, preview and photo size, etc.

StartPreview (): starts the preview and displays the preview frame data from the underlying hardware of camera on the bound surface.

StopPreview (): stops previewing, turns off the transmission of frame data at the bottom of camra, and turns off painting on surface.

Release (): release the Camera instance

TakePicture (Camera.ShutterCallback shutter, Camera.PictureCallback raw, Camera.PictureCallback jpeg): this is the main method to implement camera photography, including three callback parameters. Shutter is the callback when the shutter is pressed, raw is the callback to obtain the original photo data, and jpeg is the callback to obtain the image data compressed into jpg format.

SurfaceView: a class for drawing camera preview images, providing users with real-time preview images. Normal view and derived classes share the same surface, and all drawing must be done in the UI thread. Surfaceview is a special view, it does not share surface with other ordinary view, but holds an independent surface,surfaceview internally responsible for managing the format, size and display location of the surface. Because the UI thread has to deal with other interaction logic at the same time, the update speed and frame rate of view cannot be guaranteed, while surfaceview can be drawn in a separate thread because it holds a separate surface, so it can provide a higher frame rate. The preview image of the custom camera requires high update speed and frame rate, so it is more suitable to be displayed by surfaceview.

SurfaceHolder:surfaceholder is an abstract interface to control surface. It can control the size and format of surface, modify the pixels of surface, monitor the changes of surface and so on. The typical application of surfaceholder is used in surfaceview. Surfaceview obtains the surfaceholder instance through the getHolder () method, which manages the state of the listening surface.

SurfaceHolder.Callback API: the interface responsible for listening for surface state changes. There are three methods:

SurfaceCreated (SurfaceHolder holder): called immediately after the surface is created. When developing a custom camera, you can overload this function to call camera.open () and camera.setPreviewDisplay () to obtain camera resources, connect camera and surface, and so on.

SurfaceChanged (SurfaceHolder holder, int format, int width, int height): called when format or size changes occur in surface. When developing a custom camera, you can overload this function to call camera.startPreview to turn on the camera preview, so that the camera preview frame data can be passed to surface, thus displaying the camera preview image in real time.

SurfaceDestroyed (SurfaceHolder holder): called before the surface is destroyed. When developing a custom camera, you can overload this function by calling camera.stopPreview (), camera.release () to stop the camera preview and release camera resources.

three。 The development process of a custom camera

To customize a custom camera application, you usually need to complete the following steps, and the flow chart is shown in figure 1:

Detect and access camera resources to check whether the phone has camera resources, and if so, request access to camera resources.

Create a preview class to create a shot preview class that inherits from SurfaceView and implements the SurfaceHolder interface. This class can display a real-time preview image of the camera.

Create a preview layout with the shot preview class, you can create a layout file that combines the preview screen with the designed user interface controls.

Set up a photo listener to bind a listener to the user interface control so that it can respond to user actions (such as pressing a button) and start the photo process.

Take a picture and save the file to convert the captured image into a bitmap file, and finally output the picture saved in a variety of common formats.

Release camera resource camera is a shared resource and its life cycle must be carefully managed. When the camera is in use, the application must release it correctly to avoid conflicts when it is accessed by other programs.

Figure 1 the process of customizing a custom camera

Corresponding to code writing can be divided into three steps:

Step 1: add permission to use Camera-related features in AndroidManifest.xml. You can declare the following:

Step 2: write the camera operation function class CameraOperationHelper. The singleton mode is used to manage camera resources, encapsulate the direct call of camera API, and provide a callback interface for UI interaction with custom camera Activity. Its functions are as follows: create\ release camera, connect\ start\ close preview interface, take photos, autofocus, switch front and rear cameras, switch flash mode, and so on. For specific implementation, please refer to the official API documentation.

The third step: write a custom camera Activity, mainly customize the camera interface, achieve UI interactive logic, such as button click event handling, icon resource switching, lens size switching animation and so on. Here you need to declare a SurfaceView object to display the camera preview in real time. The connection between screen surface and camera resources is managed through SurfaceHolder and its Callback interface, and the camera preview image is displayed / closed.

four。 Some pits encountered in the development process

Here are some of the pitfalls I stepped on when developing a custom camera:

1. When Activity is set to vertical screen, the SurfaceView preview image is upside down 90 degrees.

Before explaining this problem, let's introduce the concepts of several directions on Android phones:

Screen orientation: in an Android system, the upper left corner of the screen is the origin (0P0) coordinate of the coordinate system. The extension of the origin to the right is the positive direction of the X axis and the downward extension of the origin is the positive direction of the Y axis.

Camera sensor direction: the image data of the mobile camera comes from the image sensor of the camera hardware. this sensor has a default view direction after being fixed to the phone, as shown in figure 2 below. the coordinate origin is located in the upper-left corner of the horizontal position of the phone, which is consistent with the X direction of the horizontal screen application. In other words, the X direction of the screen applied to the vertical screen is at an angle of 90 degrees.

Fig. 2 schematic diagram of camera sensor direction

Camera preview direction: because the phone screen can be rotated 360 degrees, in order to ensure that users can see the "correct" preview screen no matter how they rotate the phone (this "correct" means that the picture displayed in the UI preview interface is consistent with what the human eye sees), the bottom layer of the Android system rotates the data collected by the image sensor according to the direction of the current phone screen before sending it to the display system. Therefore, you can ensure that the preview screen is always "correct". In the camera API, you can set the camera preview direction through setDisplayOrientation (). By default, this value is 0, which is consistent with the image sensor. Therefore, for landscape applications, the preview image will not be reversed 90 degrees because the screen direction is the same as the preview direction. But for vertical screen applications, the screen direction is perpendicular to the preview direction, so it will be 90 degrees upside down. In order to get the correct preview screen, you must rotate the preview direction of the camera by 90% through API to keep it in line with the screen direction, as shown in figure 3.

Fig. 3 schematic diagram of camera preview direction

(the red arrow is the preview direction and the blue direction is the screen direction)

The direction of the camera: when the photo button is clicked, the photos taken are generated by the data collected by the image sensor directly stored on the SDCard, so the direction of the camera is consistent with the direction of the sensor.

2. SurfaceView preview image, take photos stretch deformation

Before explaining this problem, let's also talk about a few camera-related dimensions.

SurfaceView size: the size of the View used to display the camera preview image in the custom camera application, which is the size of the screen when it is full. Here, the preview image displayed by surfaceview is temporarily called mobile phone preview image.

Previewsize: the preview frame data size provided by the camera hardware. The preview frame data is transferred to SurfaceView to realize the display of the preview image. Here, the preview image corresponding to the preview frame data is temporarily called the camera preview image.

Picturesize: the frame data size provided by the camera hardware. Shooting frame data can generate a bitmap file and eventually save it into a .jpg or .png format. Here, the image corresponding to the frame data is called the camera image. Figure 4 illustrates the relationship between the above images and photos. The mobile phone preview image is the image provided directly to the user, it is generated by the camera preview image, and the photo data comes from the camera image.

Fig. 4 the relationship between several images

Here are three kinds of tensile deformation phenomena that I encountered in the development process:

1. The object is stretched and deformed in the preview screen of the mobile phone.

2. The object is stretched and deformed in the photo.

3. Click to take a picture at the moment, the phone preview screen will pause, at this time the image is stretched and deformed, and then the preview screen returns to normal after the image is restored.

The reason for phenomenon 1 is that the ratio of length to width of SurfaceView and Previewsize is different. Because the image of the mobile phone preview view is scaled by the camera preview image according to the SurfaceView size, when the aspect ratio is inconsistent, it will inevitably lead to image deformation. The reason for the latter two phenomena is the inconsistency of the aspect ratio of Previewsize and Picturesize. After checking the relevant data, it is found that the specific reason is related to the underlying implementation of some mobile camera hardware. In short, in order to avoid the occurrence of the above deformation phenomena, it is best to ensure the same ratio of length and width of SurfaceView, PreviewSize and PictureSize during development. The specific implementation can first obtain all the preview and shooting sizes supported by the camera hardware through camera.getSupportedPreviewSizes () and camera.getSupportedPictureSizes (), and then filter out the appropriate size which is consistent with the aspect ratio of SurfaceView, and update the settings through camera.setPrameters. Note: the hardware size supported by mobile cameras on the market is generally the mainstream 4:3 or 16:9, so the SurfaceView size can not be too strange, it is best to set such an aspect ratio.

3. All kinds of crash

The reasons for the first two Crash are: before focusing and taking pictures, the camera hardware must make sure that it is connected to the surface, and the camera preview is enabled, and the surface receives the preview data. If you haven't executed camera. SetPreviewDisplay or camera is not called. This runtime exception occurs when camera.autofocus or camera.takepicture is called before startPreview. In the code corresponding to the custom camera, be sure to verify that camera has set the preview Surfaceview and turned on the camera preview before executing camera.autofocus or camera.takepicture in the photo button event response. Here is a way to determine the preview status: Camera.setPreviewCallback is a callback function for preview frame data, which is called when the SurfaceView receives the camera's preview frame data, so you can set the flag bit of whether focusing and taking pictures are allowed.

It should also be noted that camera.takePicture () executes camera.stopPreview to get the frame data during execution, which shows that the preview screen is stuck, and if the user clicks the button at this time, that is, if the user calls camera.takepicture, the above crash will also appear, so during development, you may need to block the continuous clicks of the photo button.

The third crash involves image cropping. In order to support 1:1 or 4:3 size lenses, the preview view needs to be cropped. Because it is a vertical screen application, the coordinate system of the cropping region is at an angle of 90 degrees with the direction of the camera sensor, which is shown in the cropping. The x direction on the screen corresponds to the height direction on the captured image, while the y direction on the screen. Corresponding to the captured image is the width direction. Therefore, attention should be paid to the transformation of the coordinate system and cross-boundary protection in the calculation.

4. Mirroring effect of front camera

The Android camera hardware has a special setting, that is, for the front camera, the preview view is displayed with a mirror-like effect, showing a mirror image of the camera image. On the other hand, the photos taken are still taken with a camera. If you see this, you may be a little skeptical, so try the front camera on your Android phone now and compare the difference between previewing images and taking photos. This is due to the fact that the underlying camera made a horizontal flip transformation when passing the preview data of the front camera, that is, the x-direction mirror flipped 180 degrees. This change will also affect the direction of the previous vertical screen preview. For the rear camera, you can rotate 90 degrees to make the preview view correct, while for the front camera, if you also rotate 90 degrees, the preview image you see is upside down (because the x direction is flipped 180 degrees), so you must rotate another 180 degrees to display correctly, as shown in figure 5. You can understand it with the previous schematic diagram of the camera preview direction.

Fig. 5 schematic diagram of the preview direction of the front camera

In addition, because the image is not flipped horizontally, users will find that what they see with the preview is flipped left and right for the photos taken by the front camera. This will affect the user experience to some extent. In order to solve this problem, a horizontal flip matrix transformation can be added to the image captured by the front camera when generating the bitmap file.

5. The release of camera resources under the lock screen

In order to save mobile phone power and not waste camera resources, in the custom camera developed, if the preview image no longer needs to be displayed, such as pressing the Home keyboard to switch the background or lock the screen, you should turn off the preview and release the camera resources. Refer to the official API documentation. When the surfaceView becomes visible, the surface is created and the surfaceCreated callback function in the surfaceHolder.callback API is triggered. When the surfaceview becomes invisible, the surface is destroyed and the surfacedestroyed callback function is triggered. We can handle camera-related operations in the corresponding callback function, such as connecting surface and turning preview on / off. As for the release of camera resources, it can be executed in the onpause of Acticity. Accordingly, when you want to restore the preview image, you can put the camera resource application and initialization in the onResume of Acticity, and then connect camera to surface and turn on the preview by creating surfaceview.

However, in the process of development, it is found that for pressing the HOME key to cut the background scene, the program can run normally. For lock screen scenarios, crash occurs when you reapply for camera resources, saying that access to camera resources failed. So what's the reason? I added debug log to the code to check the execution order of the code, and the results are as follows:

The execution process when you press HOME on the custom camera page:

Program running-> Press the home key

The order of Activity calls is onPause- > onStop

SurfaceView called the surfaceDestroyed method

Then switch back to the program.

The order of Activity calls is onRestart- > onStart- > onResume

SurfaceView called the surfaceCreated- > surfaceChanged method

For the lock screen, the execution process is as follows:

Activity only calls the onPause method

Activity calls the onResume method after unlocking

None of the methods of surfaceholder.callback in SurfaceView are executed

The problem was found. Because the callback method of callback was not executed when locking the screen, the connection between the camera and the preview was not disconnected, and the camera resources were released, so the system reported crash when reapplying for camera resources. According to the above document, it is speculated that the off-screen lock system did not change the visibility of surfaceview, so I tried to manually set the visibile property of surfaceview in onPause and onResume, and found that the callback function could be triggered normally. Since the user should not be able to see the surfaceview when cutting the background or locking the screen, this method of manually changing the visibility of the surfaceview will not affect the user's experience.

At this point, I believe you have a deeper understanding of "what is the method of Android camera development". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.