Shulou(Shulou.com)06/01 Report--

This article mainly introduces the relevant knowledge of "what are the knowledge points of Android Camera architecture". The editor shows you the operation process through actual cases, the method of operation is simple and fast, and it is practical. I hope this article "what are the knowledge points of Android Camera architecture" can help you solve the problem.

Note: the process of this article is Camera API1 to HAL1, the code and path are based on Qualcomm platform Android 7.1 source code, API2 to HAL3 process is similar to HAL1, but API is very different, later will be added.

Overview

Take a look at the flow of App from calling Camera.open () to the HAL layer.

Warm Tip: the picture is too big, please browse by looking at the original picture

Source code location and the module to which it belongs

Frameworks/base/core/java/android/hardware/Camera.java

Camera.java is an interface called directly by App and compiled to framework.jar.

Frameworks/base/core/jni/android_hardware_Camera.cpp

This part is the JNI native part of the code called by Java, compiled into the dynamic library libandroid_runtime.so

Frameworks/av/camera/Camera.cpp

Frameworks/av/camera/CameraBase.cpp

App is connected to the Client of the CameraService part and compiled into a dynamic library libcamera_client

Frameworks/av/services/camera/libcameraservice/CameraService.cpp

Frameworks/av/services/camera/libcameraservice/CameraService.h

This is the code of CameraService. App is connected through Binder. Later, I will talk about the difficult parts of the connection process and compile it into the dynamic library libcameraservice.so.

Frameworks/av/services/camera/libcameraservice/api1/CameraClient.cpp

This part is the Client on the CameraService side, which is used to call the interface of the HAL layer, which is also compiled in libcameraservice.so.

Frameworks/av/services/camera/libcameraservice/device1/CameraHardwareInterface.h

The HAL interface defined by Google is implemented by the chip manufacturer. The above code is the Android system code written by Google, and the HAL layer and the following are the code written by the chip manufacturer.

Hardware/qcom/camera/QCamera2/HAL/QCamera2Factory.cpp

Qualcomm platform-specific code, used to create instances of different HAL versions, compiled into a dynamic library camera.xxx.so, xxx represents the platform, such as msm8953 is compiled to camera.msm8953.so

Hardware/qcom/camera/QCamera2/HAL/QCamera2HWI.cpp

Qualcomm HAL implementation code, that is, the abbreviation of QCamera2HardwareInterface, the QCamera2HardwareInterface.cpp in the above figure should be QCamera2HWI.cpp, because the source file drawn by visio has been lost, so it can not be changed, but it should be noted that the class name defined in QCamera2HWI.cpp is QCamera2HardwareInterface, and this part of the code is also compiled into the dynamic library camera.xxx.so.

Hardware/qcom/camera/QCamera2/stack/mm-camera-interface/src/mm_camera_interface.c

Qualcomm mm-camera interface, further down the HAL layer is the driver-related mm-camera.

Vendor/qcom/proprietary/mm-camera/mm-camera2/

Qualcomm mm-camera part of the code, because I do not do driver-related, not familiar with this part, do not introduce, mm-camera down is the kernel part, equivalent to the lowest level of Android.

Hardware/libhardware/hardware.c

This part of the code is used to load the HAL layer dynamic library of Camera. Before loading, the corresponding so library will be selected under several listed paths. After finding it, it will be called through dlopen to get the relevant handle.

Pay attention

In the above process, there are the following points to note:

The process of getting CameraModule

In the figure above, the initialize (CameraModule * module) function of CameraClient is called after makeClient () in CameraService.h. Camera_module_t * rawModuel is required as a parameter for CameraModule instantiation, and rawModule is associated with the dynamic library of HAL layer (camera.xxx.so) through dlopen (). Therefore, the connection between HAL layer and CameraService is actually established through dlopen (). Note that the instantiation CameraModuel process mentioned above is carried out at boot time. OnFirstRef () is called at boot time in the flow in the picture above.

Client and Service connection proc

Personally, the connection process between Client and Service is the most difficult part of the whole process. It is difficult because we are not familiar with the communication process between processes implemented by C++. Let's take a look at the connection process:

1. First call the connect () function of CameraBase.cpp in Camera.cpp

Sp Camera::connect (int cameraId, const String16& clientPackageName,int clientUid, int clientPid) {return CameraBaseT::connect (cameraId, clientPackageName, clientUid, clientPid);}

2.CameraBase.cpp connect () code

Template sp CameraBase::connect (int cameraId, const String16& clientPackageName, int clientUid, int clientPid) {ALOGV ("% s: connect", _ _ FUNCTION__); sp c = new TCam (cameraId); sp cl = new TCam Const sp& cs = getCameraService (); binder::Status ret;if (cs! = nullptr) {TCamConnectService fnConnectService = TCamTraits::fnConnectService;ret = (cs.get ()-> * fnConnectService) (cl, cameraId, clientPackageName, clientUid, clientPid, / * out*/ & c-> mCamera) } if (ret.isOk () & & c-> mCamera! = nullptr) {IInterface::asBinder (c-> mCamera)-> linkToDeath (c); c-> mStatus = NO_ERROR;} else {ALOGW ("An error occurred while connecting to camera% d:% s", cameraId, (cs! = nullptr)? "Service not available": ret.toString8 (). String (); c.clear ();} return c;}

There are two points to note in this part of the code:

1)。 TCamTraits::fnConnectService is the function pointer to the connect () function in the ICameraService class, because in Camera.cpp, it is defined as follows:

CameraTraits::TCamConnectService CameraTraits::fnConnectService = &:: android::hardware::ICameraService::connect

So (cs.get ()-> * fnConnectService) (....) Is calling the connect () function of CameraService

This part of the call is also made through Binder, and the flow chart is as follows:

Camera Connect process

The Binder interface used by the above connect process is ICameraService, and the Connect process is to get the BpCamera instance of ICamera on the client side, and the method defined by the ICamera interface is the specific operation on Camera.

It is important to note that you cannot find the ICameraService.cpp source code in the source code of Android 7.1, but it exists in Android 4.4.4 (under framework/av/camera/). This is because after ICameraService.aidl is defined in Android 7.1, the ICameraService.cpp file is automatically generated during compilation, and the resulting file is in the out directory compiled by Android:

Out/target/product/xxx/obj/SHARED_LIBRARIES/libcamera_client_intermediates/aidl-generated/src/aidl/android/hardware/ICameraService.cpp

Note: xxx indicates the product you chose when you lunch

2)。 After the first step of connection, we get an ICamera object pointer mCamera (which actually represents BpCamera, that is, the interface of the client). Later, we call other methods of Camera, such as startPreview (), through mCamera.

Client and Service invocation process

After the completion of connect, the subsequent calls to camera are also made through Binder. The API is defined in ICamera.h, and the calling process is the same as the process of connecting in the first step, that is, the basic operation of Android IPC is simply Camera.cpp-> ICamera.cpp (BpCamera)-> ICamera.cpp (BnCamera)-> CameraClient.cpp.

It is important to note that CameraClient.cpp inherits from BnCamera, which is why CameraClient can be assigned directly to ICamera in the connect () function in CameraService.

Some inheritance relationships are as follows:

In / / CameraService.h, Client inherits from BnCamera class Client: public hardware::BnCamera, public BasicClient {...}; / / CameraClient.h, CameraClient inherits from Clientclass CameraClient: public CameraService::Client {...}; Status CameraService::connect (const sp& cameraClient,int cameraId,const String16& clientPackageName,int clientUid,int clientPid,/*out*/sp* device) {/ / some code omits sp client = nullptr Ret = connectHelper (cameraClient, id,CAMERA_HAL_API_VERSION_UNSPECIFIED, clientPackageName, clientUid, clientPid, API_1,/*legacyMode*/ false,/* shimUpdateOnly*/ false,/*out*/client); / / part of the code is omitted / * client represents BnCamera, which is called to BnCamera through transact in BpCamera, and then the method * / * device = client;return ret;} defined in Client is called

As you can see, two Binder interfaces are mainly involved in the whole process, ICamera and ICameraService. The interfaces in ICamera are used to control the specific operations of Camera, such as setting parameters, enabling / closing previews, etc., while the interfaces in ICameraService are responsible for connecting to Camera and querying Camera information (getNumberOfCameras () getCameraInfo ()).

CameraService start

The above process does not explain how CameraService is started. We all know that all services in Android systems are started through init.rc, and CameraService is no exception. The relevant code path is as follows:

Frameworks/av/camera/cameraserver/

There is a cameraserver.rc under the path, which reads as follows:

Service cameraserver / system/bin/cameraserverclass mainuser cameraservergroup audio camera input drmrpcioprio rt 4writepid / dev/cpuset/camera-daemon/tasks / dev/stune/top-app/tasks

It can be seen that the executable file cameraserver is started, and this part of the code is in main_cameraserver.cpp.

Int main (int argc _ unused, char** argv _ unused) {signal (SIGPIPE, SIG_IGN); sp proc (ProcessState::self ()); sp sm = defaultServiceManager (); ALOGI ("ServiceManager:% p", sm.get ()); CameraService::instantiate (); ProcessState::self ()-> startThreadPool (); IPCThreadState::self ()-> joinThreadPool ();}

It is very simple to add a Service through the system interface

Summary of the overall architecture

As you can see from the above analysis process, the overall code of Camera is divided into the following parts:

Java API, the interface provided to App calls

JNI, because the underlying hardware and hardware are all dealing with CAccord Clippers, so JNI is essential.

Camera Client, this Client is connected to CameraService through Binder

CameraService, a stand-alone process that exists alone, starts when powered on.

Client on the CameraService side, which is called to the HAL layer through CameraHardwareInterface in this Client

HAL layer, part of the code implemented by the chip factory

The driver part contains some code of the chip factory and kernel code

Android, there are many similar hardware are used in such an architecture, basically Java-> JNI-> Service-> HAL-> driver, in the process of reading the source code, when in doubt about the process, the best way is to print some Log in the key function, actually run down the process, look at the overall architecture of the call process, understand the general process, and then pick some details as needed, so the efficiency will be higher.

This is the end of the introduction of "what are the knowledge points of Android Camera architecture". Thank you for your reading. If you want to know more about the industry, you can follow the industry information channel. The editor will update different knowledge points for you every day.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.