Skytoby
Skytoby

深入理解Android Camera架构一-应用层

Posted on

深入理解Android Camera架构一-应用层

Posted on

相机应用处于整个框架的上层，在现实生活中，为了满足各式各样的应用场景，会加入很多业务处理逻辑，但是一旦当我们拨开繁杂的业务逻辑，便会发现其核心部分依然是通过调用谷歌制订的一系列Camera Api接口来完成的，而所有的相机行为都包含在该接口中。

起初，相机系统采用的是Camera Api v1接口，它通过一个Camera 类以及该类中的几个标准方法来实现整个相机系统的预览、拍照以及录像功能，控制逻辑比较简单，同时也比较容易理解，但也正是这种简单，导致了它无法逐帧控制底层硬件，无法通过元数据进行修改进而增强帧的表达能力，再加之应用场景的多样化趋势，该接口在新功能的实现上显得些许力不从心。面对该接口难以进一步扩展相机功能这一局面，谷歌在Andorid 5.0(API Level 21)便重新对Camera进行了设计，摒弃了Camera Api v1的设计逻辑，提出了一个全新的API – camera2，引入了Session以及Request概念，将控制逻辑统一成一个视图，因此在使用上更加复杂，同时也支持了更多特性，比如逐帧控制曝光、感光度以及支持Raw格式的输出等。并且由于对控制逻辑的高度抽象化，使得该接口具有很高的灵活性，可以通过简单的操作实现30fps的全高清连拍的功能，总得来说，该接口极大地提高了对于相机框架的控制能力，同时也进一步大幅度提升了其整体性能。

谷歌提出Camera Api v2接口的同时，将其具体实现放入了Camera Framework中来完成，Framework内部负责解析来自App的请求，并且通过AIDL跨进程接口下发到Camera Service中进行处理，并且等待结果的回传。下面将通过 Camera Api v2打开Camera的流程，来介绍下如何使用该接口来控制整个相机体系。

一、打开Camera主流程

1.1 初始化TextureView并设置监听

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
 mTextureView = findViewById(R.id.textview);
 mTextureView.setSurfaceTextureListener(textureListener);
 public TextureView.SurfaceTextureListener textureListener = new TextureView.SurfaceTextureListener() {
        @Override
        public void onSurfaceTextureAvailable(SurfaceTexture surface, int width, int height) {
            //获取CameraManager服务
            mCamManager = (CameraManager) getSystemService(Context.CAMERA_SERVICE);
            try {
                //打开主摄
                mCamManager.openCamera("0", mStateCallback, null);
            } catch (CameraAccessException e) {
                e.printStackTrace();
            }
        }

        @Override
        public void onSurfaceTextureSizeChanged(SurfaceTexture surface, int width, int height) {
            log("onSurfaceTextureSizeChanged Enter");
        }

        @Override
        public boolean onSurfaceTextureDestroyed(SurfaceTexture surface) {
            log("onSurfaceTextureDestroyed Enter");
            return false;
        }

        @Override
        public void onSurfaceTextureUpdated(SurfaceTexture surface) {
            log("onSurfaceTextureUpdated Enter");
        }  
}

1.2 SurfaceTexture可用打开Camera

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
//mStateCallback获取CameraDevice
public CameraDevice.StateCallback mStateCallback = new CameraDevice.StateCallback() {
            @Override
            public void onOpened(CameraDevice camera) {
                if (camera != null) {
                    log("camera is not null");
                }
                log("onOpened");
                //返回CameraDevice,将其存入mCamDevice
                mCamDevice = camera;
                startPreview();
            }

            @Override
            public void onDisconnected(CameraDevice camera) {
                camera.close();
                mCamDevice = null;
            }

            @Override
            public void onError(CameraDevice camera, int error) {
                camera.close();
                mCamDevice = null;
            }
 };

1.3 创建Camera Session，请求Camera数据

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
//CameraDevice创建Session，请求Camera数据
public void startPreview() {
            SurfaceTexture mSurfaceTexture = mTextureView.getSurfaceTexture();
            // 设置TextureView 用于显示的缓冲区大小
            mSurfaceTexture.setDefaultBufferSize(mPreviewSize.getWidth(), mPreviewSize.getHeight());
            //创建Surface，用于显示预览数据
            mPreviewSurface = new Surface(mSurfaceTexture);
            try {
                //创建CameraCaptureSession,App需要实现CameraCaptureSession.StateCallback用于接收CameraCaptureSession实例
                mCamDevice.createCaptureSession(Arrays.asList(mPreviewSurface, mImageReader.getSurface()), new CameraCaptureSession.StateCallback() {
                    @Override
                    public void onConfigured(CameraCaptureSession session) {
                        try {
                            //创建用于预览的CaptureRequest.Builder,进而得到CaptureRequest
                            CaptureRequest.Builder b = mCamDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);
                            b.addTarget(mPreviewSurface);
                            CaptureRequest r = b.build();
                            mCamSession = session;
                            //下发预览需求
                            mCamSession.setRepeatingRequest(r, mPreviewCaptureCallback, null);
                        } catch (CameraAccessException e) {
                            e.printStackTrace();
                        }
                    }

                    CameraCaptureSession.CaptureCallback mPreviewCaptureCallback = new CameraCaptureSession.CaptureCallback() {
                        @Override
                        public void onCaptureCompleted(CameraCaptureSession session, CaptureRequest request, TotalCaptureResult result) {
                            //返回result --> meta data
                        }
                    };

                    @Override
                    public void onReady(CameraCaptureSession session) {
                        super.onReady(session);
                    }

                    @Override
                    public void onConfigureFailed(CameraCaptureSession session) {

                    }
                }, null);
            } catch (CameraAccessException e) {
                e.printStackTrace();
            }
  }

1.4 拍照

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
try {
                    //创建CaptureRequest.Builder,TEMPLATE_STILL_CAPTURE代表了此Request是用于拍照
                    CaptureRequest.Builder b = mCamDevice.createCaptureRequest(CameraDevice.TEMPLATE_STILL_CAPTURE);
                    b.addTarget(mImageReader.getSurface());
                    mCamSession.capture(b.build(), new CameraCaptureSession.CaptureCallback() {
                        @Override
                        public void onCaptureCompleted(@NonNull CameraCaptureSession session, @NonNull CaptureRequest request, @NonNull TotalCaptureResult result) {
                            super.onCaptureCompleted(session, request, result);
                            //返回result --> meta data
                        }
                    }, null);
                } catch (CameraAccessException e) {
                    e.printStackTrace();
                }
                
                
    mImageReader = ImageReader.newInstance(mCaptureSize.getWidth(), mCaptureSize.getHeight(), ImageFormat.JPEG, 2);
        mImageReader.setOnImageAvailableListener(new ImageReader.OnImageAvailableListener() {
            @Override
            public void onImageAvailable(ImageReader reader) {
                Image image = reader.acquireNextImage();
                ByteBuffer buffer = image.getPlanes()[0].getBuffer();
                byte[] bytes = new byte[buffer.remaining()];
                //由缓冲区存入字节数组
                buffer.get(bytes);
                //字节数组转换为jpeg格式图片，并存储在设备中
                doByte2JpegFile(bytes);
                image.close();
            }
        }, null /*mCameraHandler*/);

二、mCamManager.openCamera

2.1 openCamera

[->frameworks\base\core\java\android\hardware\camera2\CameraManager.java]

1
2
3
4
5
6
7
8
@RequiresPermission(android.Manifest.permission.CAMERA)
  public void openCamera(@NonNull String cameraId,
          @NonNull final CameraDevice.StateCallback callback, @Nullable Handler handler)
          throws CameraAccessException {

      openCameraForUid(cameraId, callback, CameraDeviceImpl.checkAndWrapHandler(handler),
              USE_CALLING_UID);
  }

2.2 openCameraForUid

[->frameworks\base\core\java\android\hardware\camera2\CameraManager.java]

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
public void openCameraForUid(@NonNull String cameraId,
            @NonNull final CameraDevice.StateCallback callback, @NonNull Executor executor,
            int clientUid)
            throws CameraAccessException {

        if (cameraId == null) {
            throw new IllegalArgumentException("cameraId was null");
        } else if (callback == null) {
            throw new IllegalArgumentException("callback was null");
        }
        if (CameraManagerGlobal.sCameraServiceDisabled) {
            throw new IllegalArgumentException("No cameras available on device");
        }

        openCameraDeviceUserAsync(cameraId, callback, executor, clientUid);
    }

2.3 openCameraDeviceUserAsync

[->frameworks\base\core\java\android\hardware\camera2\CameraManager.java]

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
private CameraDevice openCameraDeviceUserAsync(String cameraId,
            CameraDevice.StateCallback callback, Executor executor, final int uid)
            throws CameraAccessException {
        //获取Camera特性
        CameraCharacteristics characteristics = getCameraCharacteristics(cameraId);
        CameraDevice device = null;

        synchronized (mLock) {

            ICameraDeviceUser cameraUser = null;
            //实例化CameraDeviceImpl，传入callback
            android.hardware.camera2.impl.CameraDeviceImpl deviceImpl =
                    new android.hardware.camera2.impl.CameraDeviceImpl(
                        cameraId,
                        callback,
                        executor,
                        characteristics,
                        mContext.getApplicationInfo().targetSdkVersion);

            ICameraDeviceCallbacks callbacks = deviceImpl.getCallbacks();

            try {
                //camera2 api
                if (supportsCamera2ApiLocked(cameraId)) {
                    // Use cameraservice's cameradeviceclient implementation for HAL3.2+ devices
                    ICameraService cameraService = CameraManagerGlobal.get().getCameraService();
                    if (cameraService == null) {
                        throw new ServiceSpecificException(
                            ICameraService.ERROR_DISCONNECTED,
                            "Camera service is currently unavailable");
                    }
                    //连接设备，并将callbacks传入
                    cameraUser = cameraService.connectDevice(callbacks, cameraId,
                            mContext.getOpPackageName(), uid);
                } else {
                    // Use legacy camera implementation for HAL1 devices
                    int id;
                    try {
                        id = Integer.parseInt(cameraId);
                    } catch (NumberFormatException e) {
                        throw new IllegalArgumentException("Expected cameraId to be numeric, but it was: "
                                + cameraId);
                    }

                    Log.i(TAG, "Using legacy camera HAL.");
                    cameraUser = CameraDeviceUserShim.connectBinderShim(callbacks, id,
                            getDisplaySize());
                }
            } catch (ServiceSpecificException e) {
                if (e.errorCode == ICameraService.ERROR_DEPRECATED_HAL) {
                    throw new AssertionError("Should've gone down the shim path");
                } else if (e.errorCode == ICameraService.ERROR_CAMERA_IN_USE ||
                        e.errorCode == ICameraService.ERROR_MAX_CAMERAS_IN_USE ||
                        e.errorCode == ICameraService.ERROR_DISABLED ||
                        e.errorCode == ICameraService.ERROR_DISCONNECTED ||
                        e.errorCode == ICameraService.ERROR_INVALID_OPERATION) {
                    // Received one of the known connection errors
                    // The remote camera device cannot be connected to, so
                    // set the local camera to the startup error state
                    deviceImpl.setRemoteFailure(e);

                    if (e.errorCode == ICameraService.ERROR_DISABLED ||
                            e.errorCode == ICameraService.ERROR_DISCONNECTED ||
                            e.errorCode == ICameraService.ERROR_CAMERA_IN_USE) {
                        // Per API docs, these failures call onError and throw
                        throwAsPublicException(e);
                    }
                } else {
                    // Unexpected failure - rethrow
                    throwAsPublicException(e);
                }
            } catch (RemoteException e) {
                // Camera service died - act as if it's a CAMERA_DISCONNECTED case
                ServiceSpecificException sse = new ServiceSpecificException(
                    ICameraService.ERROR_DISCONNECTED,
                    "Camera service is currently unavailable");
                deviceImpl.setRemoteFailure(sse);
                throwAsPublicException(sse);
            }

            // TODO: factor out callback to be non-nested, then move setter to constructor
            // For now, calling setRemoteDevice will fire initial
            // onOpened/onUnconfigured callbacks.
            // This function call may post onDisconnected and throw CAMERA_DISCONNECTED if
            // cameraUser dies during setup.
            deviceImpl.setRemoteDevice(cameraUser);
            device = deviceImpl;
        }

        return device;
    }
    
     public CameraCharacteristics getCameraCharacteristics(@NonNull String cameraId)
            throws CameraAccessException {
        CameraCharacteristics characteristics = null;
        if (CameraManagerGlobal.sCameraServiceDisabled) {
            throw new IllegalArgumentException("No cameras available on device");
        }
        synchronized (mLock) {
            /*
             * Get the camera characteristics from the camera service directly if it supports it,
             * otherwise get them from the legacy shim instead.
             */
            ICameraService cameraService = CameraManagerGlobal.get().getCameraService();
            if (cameraService == null) {
                throw new CameraAccessException(CameraAccessException.CAMERA_DISCONNECTED,
                        "Camera service is currently unavailable");
            }
            try {
                Size displaySize = getDisplaySize();

                // First check isHiddenPhysicalCamera to avoid supportsCamera2ApiLocked throwing
                // exception in case cameraId is a hidden physical camera.
                if (!isHiddenPhysicalCamera(cameraId) && !supportsCamera2ApiLocked(cameraId)) {
                    // Legacy backwards compatibility path; build static info from the camera
                    // parameters
                    int id = Integer.parseInt(cameraId);

                    String parameters = cameraService.getLegacyParameters(id);

                    CameraInfo info = cameraService.getCameraInfo(id);

                    characteristics = LegacyMetadataMapper.createCharacteristics(parameters, info,
                            id, displaySize);
                } else {
                    // Normal path: Get the camera characteristics directly from the camera service
                    CameraMetadataNative info = cameraService.getCameraCharacteristics(cameraId);
                    try {
                        info.setCameraId(Integer.parseInt(cameraId));
                    } catch (NumberFormatException e) {
                        Log.e(TAG, "Failed to parse camera Id " + cameraId + " to integer");
                    }
                    info.setDisplaySize(displaySize);

                    characteristics = new CameraCharacteristics(info);
                }
            } catch (ServiceSpecificException e) {
                throwAsPublicException(e);
            } catch (RemoteException e) {
                // Camera service died - act as if the camera was disconnected
                throw new CameraAccessException(CameraAccessException.CAMERA_DISCONNECTED,
                        "Camera service is currently unavailable", e);
            }
        }
        return characteristics;
    }

2.3.1 创建CameraDeviceImpl对象

[->frameworks\base\core\java\android\hardware\camera2\impl\CameraDeviceImpl.java]

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
public CameraDeviceImpl(String cameraId, StateCallback callback, Executor executor,
                    CameraCharacteristics characteristics, int appTargetSdkVersion) {
    if (cameraId == null || callback == null || executor == null || characteristics == null) {
        throw new IllegalArgumentException("Null argument given");
    }
    mCameraId = cameraId;
    mDeviceCallback = callback;
    mDeviceExecutor = executor;
    mCharacteristics = characteristics;
    mAppTargetSdkVersion = appTargetSdkVersion;

    final int MAX_TAG_LEN = 23;
    String tag = String.format("CameraDevice-JV-%s", mCameraId);
    if (tag.length() > MAX_TAG_LEN) {
        tag = tag.substring(0, MAX_TAG_LEN);
    }
    TAG = tag;

    Integer partialCount =
            mCharacteristics.get(CameraCharacteristics.REQUEST_PARTIAL_RESULT_COUNT);
    if (partialCount == null) {
        // 1 means partial result is not supported.
        mTotalPartialCount = 1;
    } else {
        mTotalPartialCount = partialCount;
    }
    mIsPrivilegedApp = checkPrivilegedAppList();
}

2.3.2 deviceImpl.getCallbacks

获取CameraDeviceCallbacks，用于下面的connectDevice传入，mCallbacks获取设备的状态，并将这些状态转发给StateCallback，从而同步给应用。mCallbacks起到了代理作用，其实现了ICameraDeviceCallbacks.Stub接口

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
    private final CameraDeviceCallbacks mCallbacks = new CameraDeviceCallbacks();
    
    public CameraDeviceCallbacks getCallbacks() {
        return mCallbacks;
    }
    
    public class CameraDeviceCallbacks extends ICameraDeviceCallbacks.Stub {

        @Override
        public IBinder asBinder() {
            return this;
        }

        @Override
        public void onDeviceError(final int errorCode, CaptureResultExtras resultExtras) {
            if (DEBUG) {
                Log.d(TAG, String.format(
                        "Device error received, code %d, frame number %d, request ID %d, subseq ID %d",
                        errorCode, resultExtras.getFrameNumber(), resultExtras.getRequestId(),
                        resultExtras.getSubsequenceId()));
            }

            synchronized(mInterfaceLock) {
                if (mRemoteDevice == null) {
                    return; // Camera already closed
                }

                switch (errorCode) {
                    case ERROR_CAMERA_DISCONNECTED:
                        final long ident = Binder.clearCallingIdentity();
                        try {
                            CameraDeviceImpl.this.mDeviceExecutor.execute(mCallOnDisconnected);
                        } finally {
                            Binder.restoreCallingIdentity(ident);
                        }
                        break;
                    case ERROR_CAMERA_REQUEST:
                    case ERROR_CAMERA_RESULT:
                    case ERROR_CAMERA_BUFFER:
                        onCaptureErrorLocked(errorCode, resultExtras);
                        break;
                    case ERROR_CAMERA_DEVICE:
                        scheduleNotifyError(StateCallback.ERROR_CAMERA_DEVICE);
                        break;
                    case ERROR_CAMERA_DISABLED:
                        scheduleNotifyError(StateCallback.ERROR_CAMERA_DISABLED);
                        break;
                    default:
                        Log.e(TAG, "Unknown error from camera device: " + errorCode);
                        scheduleNotifyError(StateCallback.ERROR_CAMERA_SERVICE);
                }
            }
        }
        ....
}

2.3.3 getCameraService

获取cameraService，通过CameraManager中的CameraManagerGlobal类，再通过ServiceManager获取ICameraService实例，代理CameraService。如何通过CAMERA_SERVICE_BINDER_NAME获取getCameraService代理，将在后面的章节进行详细说明

[->frameworks\base\core\java\android\hardware\camera2\CameraManager.java]

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
// Use cameraservice's cameradeviceclient implementation for HAL3.2+ devices
// frameworks/av/camera/aidl/android/hardware/ICameraService.aidl
ICameraService cameraService = CameraManagerGlobal.get().getCameraService();

public ICameraService getCameraService() {
         synchronized(mLock) {
             connectCameraServiceLocked();
             if (mCameraService == null && !sCameraServiceDisabled) {
                 Log.e(TAG, "Camera service is unavailable");
             }
             return mCameraService;
         }
  }
  
   private void connectCameraServiceLocked() {
         // Only reconnect if necessary
         if (mCameraService != null || sCameraServiceDisabled) return;

         Log.i(TAG, "Connecting to camera service");
         //获取cameraServiceBinder服务
         IBinder cameraServiceBinder = ServiceManager.getService(CAMERA_SERVICE_BINDER_NAME);
         if (cameraServiceBinder == null) {
             // Camera service is now down, leave mCameraService as null
             return;
         }
         try {
             cameraServiceBinder.linkToDeath(this, /*flags*/ 0);
         } catch (RemoteException e) {
             // Camera service is now down, leave mCameraService as null
             return;
         }
         //将binder转化为ICameraService实例
         ICameraService cameraService = ICameraService.Stub.asInterface(cameraServiceBinder);

         try {
             CameraMetadataNative.setupGlobalVendorTagDescriptor();
         } catch (ServiceSpecificException e) {
             handleRecoverableSetupErrors(e);
         }

         try {
             CameraStatus[] cameraStatuses = cameraService.addListener(this);
             for (CameraStatus c : cameraStatuses) {
                 onStatusChangedLocked(c.status, c.cameraId);
             }
             mCameraService = cameraService;
         } catch(ServiceSpecificException e) {
             // Unexpected failure
             throw new IllegalStateException("Failed to register a camera service listener", e);
         } catch (RemoteException e) {
             // Camera service is now down, leave mCameraService as null
         }
     }

2.3.4 connectDevice

通过CameraService获取ICameraDeviceUser代理类，上面所有的代理实现aidl在源代码frameworks/av/camera/aidl/android/hardware/camera2中可以找到

1
2
3
4
5
6
7
8
9
10
11
12
//frameworks/av/camera/aidl/android/hardware/camera2/ICameraDeviceUser.aidl
ICameraDeviceUser cameraUser = null;
cameraUser = cameraService.connectDevice(callbacks, cameraId,
                            mContext.getOpPackageName(), uid);
 /**
     * Open a camera device through the new camera API
     * Only supported for device HAL versions >= 3.2
     */
ICameraDeviceUser connectDevice(ICameraDeviceCallbacks callbacks,
            String cameraId,
            String opPackageName,
            int clientUid);

2.3.5 setRemoteDevice

将获取到的ICameraDeviceUser保存到CameraDeviceImpl用于管理

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
deviceImpl.setRemoteDevice(cameraUser);
public void setRemoteDevice(ICameraDeviceUser remoteDevice) throws CameraAccessException {
      synchronized(mInterfaceLock) {
          // TODO: Move from decorator to direct binder-mediated exceptions
          // If setRemoteFailure already called, do nothing
          if (mInError) return;

          mRemoteDevice = new ICameraDeviceUserWrapper(remoteDevice);

          IBinder remoteDeviceBinder = remoteDevice.asBinder();
          // For legacy camera device, remoteDevice is in the same process, and
          // asBinder returns NULL.
          if (remoteDeviceBinder != null) {
              try {
                  remoteDeviceBinder.linkToDeath(this, /*flag*/ 0);
              } catch (RemoteException e) {
                  CameraDeviceImpl.this.mDeviceExecutor.execute(mCallOnDisconnected);

                  throw new CameraAccessException(CameraAccessException.CAMERA_DISCONNECTED,
                          "The camera device has encountered a serious error");
              }
          }

          mDeviceExecutor.execute(mCallOnOpened);
          mDeviceExecutor.execute(mCallOnUnconfigured);
      }
  }

2.3.6 mDeviceExecutor.execute

通过StateCallback返回给应用CameraDeviceImpl，用于后续的操作

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
private final Runnable mCallOnOpened = new Runnable() {
       @Override
       public void run() {
           StateCallbackKK sessionCallback = null;
           synchronized(mInterfaceLock) {
               if (mRemoteDevice == null) return; // Camera already closed

               sessionCallback = mSessionStateCallback;
           }
           if (sessionCallback != null) {
               sessionCallback.onOpened(CameraDeviceImpl.this);
           }
           mDeviceCallback.onOpened(CameraDeviceImpl.this);
       }
   };

2.4 小结

当应用打开相机应用时，会去调用该方法打开一个相机设备，其中该方法最终经过层层调用会调用到Camera Framework中的openCameraDeviceUserAsync方法，在该方法中主要做了三件事：

  1. 首先是获取ICameraService代理，调用其getCameraInfo方法获取当前设备的属性。
  2. 其次是实例化了一个CameraDeviceImpl对象，并将来自App的CameraDevice.StateCallback接口存入该对象中，再将CameraDeviceImpl中的内部类CameraDeviceCallback作为参数通过ICameraService的connectDevice方法传入Camera Service去打开并获取一个ICameraDeviceUser代理，并将该代理存入CameraDeviceImpl中进行管理。
  3. 最后通过App传入的回调将CameraDeviceImpl返回给App使用，至此整个流程便完成了。

三、mCamDevice.createCaptureSession

3.1 createCaptureSession

1
2
3
4
5
6
7
8
9
10
11
public void createCaptureSession(List<Surface> outputs,
            CameraCaptureSession.StateCallback callback, Handler handler)
            throws CameraAccessException {
        List<OutputConfiguration> outConfigurations = new ArrayList<>(outputs.size());
        for (Surface surface : outputs) {
            outConfigurations.add(new OutputConfiguration(surface));
        }
        createCaptureSessionInternal(null, outConfigurations, callback,
                checkAndWrapHandler(handler), /*operatingMode*/ICameraDeviceUser.NORMAL_MODE,
                /*sessionParams*/ null);
    }

3.2 createCaptureSessionInternal

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
private void createCaptureSessionInternal(InputConfiguration inputConfig,
            List<OutputConfiguration> outputConfigurations,
            CameraCaptureSession.StateCallback callback, Executor executor,
            int operatingMode, CaptureRequest sessionParams) throws CameraAccessException {
        synchronized(mInterfaceLock) {
            if (DEBUG) {
                Log.d(TAG, "createCaptureSessionInternal");
            }

            checkIfCameraClosedOrInError();

            boolean isConstrainedHighSpeed =
                    (operatingMode == ICameraDeviceUser.CONSTRAINED_HIGH_SPEED_MODE);
            if (isConstrainedHighSpeed && inputConfig != null) {
                throw new IllegalArgumentException("Constrained high speed session doesn't support"
                        + " input configuration yet.");
            }

            // Notify current session that it's going away, before starting camera operations
            // After this call completes, the session is not allowed to call into CameraDeviceImpl
            if (mCurrentSession != null) {
                mCurrentSession.replaceSessionClose();
            }

            // TODO: dont block for this
            boolean configureSuccess = true;
            CameraAccessException pendingException = null;
            Surface input = null;
            try {
                // configure streams and then block until IDLE
                configureSuccess = configureStreamsChecked(inputConfig, outputConfigurations,
                        operatingMode, sessionParams);
                if (configureSuccess == true && inputConfig != null) {
                    input = mRemoteDevice.getInputSurface();
                }
            } catch (CameraAccessException e) {
                configureSuccess = false;
                pendingException = e;
                input = null;
                if (DEBUG) {
                    Log.v(TAG, "createCaptureSession - failed with exception ", e);
                }
            }

            // Fire onConfigured if configureOutputs succeeded, fire onConfigureFailed otherwise.
            CameraCaptureSessionCore newSession = null;
            if (isConstrainedHighSpeed) {
                ArrayList<Surface> surfaces = new ArrayList<>(outputConfigurations.size());
                for (OutputConfiguration outConfig : outputConfigurations) {
                    surfaces.add(outConfig.getSurface());
                }
                StreamConfigurationMap config =
                    getCharacteristics().get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);
                SurfaceUtils.checkConstrainedHighSpeedSurfaces(surfaces, /*fpsRange*/null, config);

                newSession = new CameraConstrainedHighSpeedCaptureSessionImpl(mNextSessionId++,
                        callback, executor, this, mDeviceExecutor, configureSuccess,
                        mCharacteristics);
            } else {
                newSession = new CameraCaptureSessionImpl(mNextSessionId++, input,
                        callback, executor, this, mDeviceExecutor, configureSuccess);
            }

            // TODO: wait until current session closes, then create the new session
            mCurrentSession = newSession;

            if (pendingException != null) {
                throw pendingException;
            }

            mSessionStateCallback = mCurrentSession.getDeviceStateCallback();
        }
    }

3.2.1 configureStreamsChecked

通过mRemoteDevice配置数据流，mRemoteDevice是在2.3.5中setRemoteDevice初始化，最终调用的是ICameraDeviceUser接口进行deleteStream、createStream等操作。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
public boolean configureStreamsChecked(InputConfiguration inputConfig,
            List<OutputConfiguration> outputs, int operatingMode, CaptureRequest sessionParams)
                    throws CameraAccessException {
        // Treat a null input the same an empty list
        if (outputs == null) {
            outputs = new ArrayList<OutputConfiguration>();
        }
        if (outputs.size() == 0 && inputConfig != null) {
            throw new IllegalArgumentException("cannot configure an input stream without " +
                    "any output streams");
        }

        checkInputConfiguration(inputConfig);

        boolean success = false;

        synchronized(mInterfaceLock) {
            checkIfCameraClosedOrInError();
            // Streams to create
            HashSet<OutputConfiguration> addSet = new HashSet<OutputConfiguration>(outputs);
            // Streams to delete
            List<Integer> deleteList = new ArrayList<Integer>();

            // Determine which streams need to be created, which to be deleted
            for (int i = 0; i < mConfiguredOutputs.size(); ++i) {
                int streamId = mConfiguredOutputs.keyAt(i);
                OutputConfiguration outConfig = mConfiguredOutputs.valueAt(i);

                if (!outputs.contains(outConfig) || outConfig.isDeferredConfiguration()) {
                    // Always delete the deferred output configuration when the session
                    // is created, as the deferred output configuration doesn't have unique surface
                    // related identifies.
                    deleteList.add(streamId);
                } else {
                    addSet.remove(outConfig);  // Don't create a stream previously created
                }
            }

            mDeviceExecutor.execute(mCallOnBusy);
            stopRepeating();

            try {
                waitUntilIdle();

                mRemoteDevice.beginConfigure();

                // reconfigure the input stream if the input configuration is different.
                InputConfiguration currentInputConfig = mConfiguredInput.getValue();
                if (inputConfig != currentInputConfig &&
                        (inputConfig == null || !inputConfig.equals(currentInputConfig))) {
                    if (currentInputConfig != null) {
                        mRemoteDevice.deleteStream(mConfiguredInput.getKey());
                        mConfiguredInput = new SimpleEntry<Integer, InputConfiguration>(
                                REQUEST_ID_NONE, null);
                    }
                    if (inputConfig != null) {
                        int streamId = mRemoteDevice.createInputStream(inputConfig.getWidth(),
                                inputConfig.getHeight(), inputConfig.getFormat());
                        mConfiguredInput = new SimpleEntry<Integer, InputConfiguration>(
                                streamId, inputConfig);
                    }
                }

                // Delete all streams first (to free up HW resources)
                for (Integer streamId : deleteList) {
                    mRemoteDevice.deleteStream(streamId);
                    mConfiguredOutputs.delete(streamId);
                }

                // Add all new streams
                for (OutputConfiguration outConfig : outputs) {
                    if (addSet.contains(outConfig)) {
                        int streamId = mRemoteDevice.createStream(outConfig);
                        mConfiguredOutputs.put(streamId, outConfig);
                    }
                }
                operatingMode = (operatingMode | (customOpMode << 16));

                if (sessionParams != null) {
                    mRemoteDevice.endConfigure(operatingMode, sessionParams.getNativeCopy());
                } else {
                    mRemoteDevice.endConfigure(operatingMode, null);
                }

                success = true;
            } catch (IllegalArgumentException e) {
                // OK. camera service can reject stream config if it's not supported by HAL
                // This is only the result of a programmer misusing the camera2 api.
                Log.w(TAG, "Stream configuration failed due to: " + e.getMessage());
                return false;
            } catch (CameraAccessException e) {
                if (e.getReason() == CameraAccessException.CAMERA_IN_USE) {
                    throw new IllegalStateException("The camera is currently busy." +
                            " You must wait until the previous operation completes.", e);
                }
                throw e;
            } finally {
                if (success && outputs.size() > 0) {
                    mDeviceExecutor.execute(mCallOnIdle);
                } else {
                    // Always return to the 'unconfigured' state if we didn't hit a fatal error
                    mDeviceExecutor.execute(mCallOnUnconfigured);
                }
            }
        }

        return success;
    }

3.2.2 创建CameraCaptureSessionImpl

创建CameraCaptureSessionImpl，并传入CameraCaptureSession.StateCallback用于返回给应用侧

1
2
newSession = new CameraCaptureSessionImpl(mNextSessionId++, input,
                       callback, executor, this, mDeviceExecutor, configureSuccess);

将应用侧mStateCallback保存在CameraCaptureSessionImpl，用于结果的回调。配置流成功，返回给应用侧状态mStateCallback.onConfigured

[->frameworks\base\core\java\android\hardware\camera2\impl\CameraCaptureSessionImpl.java]

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
CameraCaptureSessionImpl(int id, Surface input,
            CameraCaptureSession.StateCallback callback, Executor stateExecutor,
            android.hardware.camera2.impl.CameraDeviceImpl deviceImpl,
            Executor deviceStateExecutor, boolean configureSuccess) {
        if (callback == null) {
            throw new IllegalArgumentException("callback must not be null");
        }

        mId = id;
        mIdString = String.format("Session %d: ", mId);

        mInput = input;
        mStateExecutor = checkNotNull(stateExecutor, "stateExecutor must not be null");
        mStateCallback = createUserStateCallbackProxy(mStateExecutor, callback);

        mDeviceExecutor = checkNotNull(deviceStateExecutor,
                "deviceStateExecutor must not be null");
        mDeviceImpl = checkNotNull(deviceImpl, "deviceImpl must not be null");

        /*
         * Use the same handler as the device's StateCallback for all the internal coming events
         *
         * This ensures total ordering between CameraDevice.StateCallback and
         * CameraDeviceImpl.CaptureCallback events.
         */
        mSequenceDrainer = new TaskDrainer<>(mDeviceExecutor, new SequenceDrainListener(),
                /*name*/"seq");
        mIdleDrainer = new TaskSingleDrainer(mDeviceExecutor, new IdleDrainListener(),
                /*name*/"idle");
        mAbortDrainer = new TaskSingleDrainer(mDeviceExecutor, new AbortDrainListener(),
                /*name*/"abort");

        // CameraDevice should call configureOutputs and have it finish before constructing us

        if (configureSuccess) {
            mStateCallback.onConfigured(this);
            if (DEBUG) Log.v(TAG, mIdString + "Created session successfully");
            mConfigureSuccess = true;
        } else {
            mStateCallback.onConfigureFailed(this);
            mClosed = true; // do not fire any other callbacks, do not allow any other work
            Log.e(TAG, mIdString + "Failed to create capture session; configuration failed");
            mConfigureSuccess = false;
        }
    }

3.3 小结

在打开相机设备之后便需要去创建一个相机会话，用于传输图像请求，其最终实现是调用该方法来进行实现的，而该方法会去调用到Camera Framework中的createCaptureSessionInternal方法，该方法主要做了两件事：

  1. 首先调用configureStreamsChecked方法来配置数据流。
  2. 其次实例化了一个CameraCaptureImpl对象，并通过传入CameraCaptureSession.StateCallback回调类将该对象发送至至App中。

而在configureStreamsChecked方法中会去调用ICameraDeviceUser代理的一系列方法进行数据流配置，其中调用cancelRequest方法停掉当前的预览流程，调用deleteStream方法删除之前的数据流，调用createStream创建新的数据流，最后调用endConfigure来进行数据流的配置工作，针对性的配置便在最后这个endConfigure方法中。

四、mCamDevice.createCaptureRequest

[->frameworks/base/core/java/android/hardware/camera2/CaptureRequest.java ]

1
2
3
CaptureRequest.Builder b = mCamDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);
b.addTarget(mPreviewSurface);
CaptureRequest r = b.build();

请求类型有6种

TEMPLATE_PREVIEW : 创建预览的请求
TEMPLATE_STILL_CAPTURE: 创建一个适合于静态图像捕获的请求，图像质量优先于帧速率
TEMPLATE_RECORD : 创建视频录制的请求
TEMPLATE_VIDEO_SNAPSHOT : 创建视视频录制时截屏的请求
TEMPLATE_ZERO_SHUTTER_LAG : 创建一个适用于零快门延迟的请求。在不影响预览帧率的情况下最大化图像质量
TEMPLATE_MANUAL : 创建一个基本捕获请求，这种请求中所有的自动控制都是禁用的(自动曝光，自动白平衡、自动焦点)

4.1 createCaptureRequest

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
public CaptureRequest.Builder createCaptureRequest(int templateType,
           Set<String> physicalCameraIdSet)
           throws CameraAccessException {
       synchronized(mInterfaceLock) {
           checkIfCameraClosedOrInError();

           for (String physicalId : physicalCameraIdSet) {
               if (physicalId == getId()) {
                   throw new IllegalStateException("Physical id matches the logical id!");
               }
           }

           CameraMetadataNative templatedRequest = null;

           templatedRequest = mRemoteDevice.createDefaultRequest(templateType);

           // If app target SDK is older than O, or it's not a still capture template, enableZsl
           // must be false in the default request.
           if (mAppTargetSdkVersion < Build.VERSION_CODES.O ||
                   templateType != TEMPLATE_STILL_CAPTURE) {
               overrideEnableZsl(templatedRequest, false);
           }

           CaptureRequest.Builder builder = new CaptureRequest.Builder(
                   templatedRequest, /*reprocess*/false, CameraCaptureSession.SESSION_ID_NONE,
                   getId(), physicalCameraIdSet);

           return builder;
       }
   }
   
   public CameraMetadataNative createDefaultRequest(int templateId) throws CameraAccessException {
       try {
           return mRemoteDevice.createDefaultRequest(templateId);
       } catch (Throwable t) {
           CameraManager.throwAsPublicException(t);
           throw new UnsupportedOperationException("Unexpected exception", t);
       }
   }
   
  public Builder(CameraMetadataNative template, boolean reprocess,
               int reprocessableSessionId, String logicalCameraId,
               Set<String> physicalCameraIdSet) {
           mRequest = new CaptureRequest(template, reprocess, reprocessableSessionId,
                   logicalCameraId, physicalCameraIdSet);
  }

4.2 小结

在创建并获取相机会话之后，便可以开始下发图像请求了，而在此之前，需要通过该方法来创建一个CaptureRequest，一旦调用该方法，最终会调用到Camera Service中ICameraDeviceUser的createDefaultRequest方法来创建一个默认配置的CameraMetadataNative，其次实例化一个CaptureRequest.Builder对象，并将刚才获取的CameraMetadataNative传入其中，之后返回该CaptureRequest.Builder对象，在App中，直接通过调用该Buidler对象的build方法，获取一个CaptureRequest对象。

CaptureRequest对象也创建成功了，接下来需要下发图像请求了，一般常用请求分为两种,一个是预览一个是拍照。

五、mCamSession.setRepeatingRequest

5.1 setRepeatingRequest

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
public int setRepeatingRequest(CaptureRequest request, CaptureCallback callback,
         Handler handler) throws CameraAccessException {
     checkRepeatingRequest(request);

     synchronized (mDeviceImpl.mInterfaceLock) {
         checkNotClosed();

         handler = checkHandler(handler, callback);

         if (DEBUG) {
             Log.v(TAG, mIdString + "setRepeatingRequest - request " + request + ", callback " +
                     callback + " handler" + " " + handler);
         }

         return addPendingSequence(mDeviceImpl.setRepeatingRequest(request,
                 createCaptureCallbackProxy(handler, callback), mDeviceExecutor));
     }
 }

5.2 addPendingSequence

1
2
3
4
private int addPendingSequence(int sequenceId) {
      mSequenceDrainer.taskStarted(sequenceId);
      return sequenceId;
  }

5.2.1 setRepeatingRequest

1
2
3
4
5
6
public int setRepeatingRequest(CaptureRequest request, CaptureCallback callback,
         Executor executor) throws CameraAccessException {
     List<CaptureRequest> requestList = new ArrayList<CaptureRequest>();
     requestList.add(request);
     return submitCaptureRequest(requestList, callback, executor, /*streaming*/true);
 }

5.2.2 submitRequestList

最后调用的是ICameraDeviceUser的submitRequestList方法

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
private int submitCaptureRequest(List<CaptureRequest> requestList, CaptureCallback callback,
            Executor executor, boolean repeating) throws CameraAccessException {

        // Need a valid executor, or current thread needs to have a looper, if
        // callback is valid
        executor = checkExecutor(executor, callback);

        synchronized(mInterfaceLock) {
            checkIfCameraClosedOrInError();

            // Make sure that there all requests have at least 1 surface; all surfaces are non-null;
            // the surface isn't a physical stream surface for reprocessing request
            for (CaptureRequest request : requestList) {
                if (request.getTargets().isEmpty()) {
                    throw new IllegalArgumentException(
                            "Each request must have at least one Surface target");
                }

                for (Surface surface : request.getTargets()) {
                    if (surface == null) {
                        throw new IllegalArgumentException("Null Surface targets are not allowed");
                    }

                    for (int i = 0; i < mConfiguredOutputs.size(); i++) {
                        OutputConfiguration configuration = mConfiguredOutputs.valueAt(i);
                        if (configuration.isForPhysicalCamera()
                                && configuration.getSurfaces().contains(surface)) {
                            if (request.isReprocess()) {
                                throw new IllegalArgumentException(
                                        "Reprocess request on physical stream is not allowed");
                            }
                        }
                    }
                }
            }

            if (repeating) {
                stopRepeating();
            }

            SubmitInfo requestInfo;

            CaptureRequest[] requestArray = requestList.toArray(new CaptureRequest[requestList.size()]);
            // Convert Surface to streamIdx and surfaceIdx
            for (CaptureRequest request : requestArray) {
                request.convertSurfaceToStreamId(mConfiguredOutputs);
            }

            requestInfo = mRemoteDevice.submitRequestList(requestArray, repeating);
            if (DEBUG) {
                Log.v(TAG, "last frame number " + requestInfo.getLastFrameNumber());
            }

            for (CaptureRequest request : requestArray) {
                request.recoverStreamIdToSurface();
            }
            //保存callback到CameraDeviceImpl中
            if (callback != null) {
                mCaptureCallbackMap.put(requestInfo.getRequestId(),
                        new CaptureCallbackHolder(
                            callback, requestList, executor, repeating, mNextSessionId - 1));
            } else {
                if (DEBUG) {
                    Log.d(TAG, "Listen for request " + requestInfo.getRequestId() + " is null");
                }
            }

            if (repeating) {
                if (mRepeatingRequestId != REQUEST_ID_NONE) {
                    checkEarlyTriggerSequenceComplete(mRepeatingRequestId,
                            requestInfo.getLastFrameNumber(),
                            mRepeatingRequestTypes);
                }
                mRepeatingRequestId = requestInfo.getRequestId();
                mRepeatingRequestTypes = getRequestTypes(requestArray);
            } else {
                mRequestLastFrameNumbersList.add(
                    new RequestLastFrameNumbersHolder(requestList, requestInfo));
            }

            if (mIdle) {
                mDeviceExecutor.execute(mCallOnActive);
            }
            mIdle = false;

            return requestInfo.getRequestId();
        }
    }

5.3 小结

应用调用该方法开始预览流程，通过层层调用最终会调用到Framework中的submitCaptureRequest方法，该方法主要做了两件事：

  1. 首先调用CameraService层CameraDeviceUser的submitRequestList方法，将此次Request下发到CameraService中。
  2. 其次将App通过参数传入的CameraCaptureSession.CaptureCallback对象存到CameraDeviceImpI对象中。

六、mCamSession.capture

6.1 capture

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
public int capture(CaptureRequest request, CaptureCallback callback,
            Handler handler) throws CameraAccessException {
        checkCaptureRequest(request);

        synchronized (mDeviceImpl.mInterfaceLock) {
            checkNotClosed();

            handler = checkHandler(handler, callback);

            if (DEBUG) {
                Log.v(TAG, mIdString + "capture - request " + request + ", callback " + callback +
                        " handler " + handler);
            }

            return addPendingSequence(mDeviceImpl.capture(request,
                    createCaptureCallbackProxy(handler, callback), mDeviceExecutor));
        }
    }

6.2 mDeviceImpl.capture

1
2
3
4
5
6
7
8
9
public int capture(CaptureRequest request, CaptureCallback callback, Executor executor)
           throws CameraAccessException {
       if (DEBUG) {
           Log.d(TAG, "calling capture");
       }
       List<CaptureRequest> requestList = new ArrayList<CaptureRequest>();
       requestList.add(request);
       return submitCaptureRequest(requestList, callback, executor, /*streaming*/false);
   }

6.3 submitCaptureRequest

该过程和5.3.1类似，最后submitRequestList提交拍照请求

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
private int submitCaptureRequest(List<CaptureRequest> requestList, CaptureCallback callback,
            Executor executor, boolean repeating) throws CameraAccessException {

        // Need a valid executor, or current thread needs to have a looper, if
        // callback is valid
        executor = checkExecutor(executor, callback);

        synchronized(mInterfaceLock) {
            checkIfCameraClosedOrInError();

            // Make sure that there all requests have at least 1 surface; all surfaces are non-null;
            // the surface isn't a physical stream surface for reprocessing request
            for (CaptureRequest request : requestList) {
                if (request.getTargets().isEmpty()) {
                    throw new IllegalArgumentException(
                            "Each request must have at least one Surface target");
                }

                for (Surface surface : request.getTargets()) {
                    if (surface == null) {
                        throw new IllegalArgumentException("Null Surface targets are not allowed");
                    }

                    for (int i = 0; i < mConfiguredOutputs.size(); i++) {
                        OutputConfiguration configuration = mConfiguredOutputs.valueAt(i);
                        if (configuration.isForPhysicalCamera()
                                && configuration.getSurfaces().contains(surface)) {
                            if (request.isReprocess()) {
                                throw new IllegalArgumentException(
                                        "Reprocess request on physical stream is not allowed");
                            }
                        }
                    }
                }
            }

            if (repeating) {
                stopRepeating();
            }

            SubmitInfo requestInfo;

            CaptureRequest[] requestArray = requestList.toArray(new CaptureRequest[requestList.size()]);
            // Convert Surface to streamIdx and surfaceIdx
            for (CaptureRequest request : requestArray) {
                request.convertSurfaceToStreamId(mConfiguredOutputs);
            }

            requestInfo = mRemoteDevice.submitRequestList(requestArray, repeating);
            if (DEBUG) {
                Log.v(TAG, "last frame number " + requestInfo.getLastFrameNumber());
            }

            for (CaptureRequest request : requestArray) {
                request.recoverStreamIdToSurface();
            }

            if (callback != null) {
                mCaptureCallbackMap.put(requestInfo.getRequestId(),
                        new CaptureCallbackHolder(
                            callback, requestList, executor, repeating, mNextSessionId - 1));
            } else {
                if (DEBUG) {
                    Log.d(TAG, "Listen for request " + requestInfo.getRequestId() + " is null");
                }
            }

            if (repeating) {
                if (mRepeatingRequestId != REQUEST_ID_NONE) {
                    checkEarlyTriggerSequenceComplete(mRepeatingRequestId,
                            requestInfo.getLastFrameNumber(),
                            mRepeatingRequestTypes);
                }
                mRepeatingRequestId = requestInfo.getRequestId();
                mRepeatingRequestTypes = getRequestTypes(requestArray);
            } else {
                mRequestLastFrameNumbersList.add(
                    new RequestLastFrameNumbersHolder(requestList, requestInfo));
            }

            if (mIdle) {
                mDeviceExecutor.execute(mCallOnActive);
            }
            mIdle = false;

            return requestInfo.getRequestId();
        }
    }

6.4 onCaptureProgressed、onCaptureCompleted

一旦Request下发到Camera Service之后，当底层生成了Partial Meta Data数据，Camera Service会调用通过调用在打开相机设备时传入的ICameraDeviceCallback代理，通过其onResultReceived方法将数据传回Framework，之后调用App传入的CameraCaptureSession.CaptureCallback中的onCaputreProgressed、onCaptureCompleted方法将结果回传至App进行解析以及后处理。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
public class CameraDeviceCallbacks extends ICameraDeviceCallbacks.Stub {
  .....
 public void onResultReceived(CameraMetadataNative result,
                CaptureResultExtras resultExtras, PhysicalCaptureResultInfo physicalResults[])
                throws RemoteException {

            int requestId = resultExtras.getRequestId();
            long frameNumber = resultExtras.getFrameNumber();

            if (DEBUG) {
                Log.v(TAG, "Received result frame " + frameNumber + " for id "
                        + requestId);
            }

            synchronized(mInterfaceLock) {
                if (mRemoteDevice == null) return; // Camera already closed

                // TODO: Handle CameraCharacteristics access from CaptureResult correctly.
                result.set(CameraCharacteristics.LENS_INFO_SHADING_MAP_SIZE,
                        getCharacteristics().get(CameraCharacteristics.LENS_INFO_SHADING_MAP_SIZE));

                final CaptureCallbackHolder holder =
                        CameraDeviceImpl.this.mCaptureCallbackMap.get(requestId);
                final CaptureRequest request = holder.getRequest(resultExtras.getSubsequenceId());

                boolean isPartialResult =
                        (resultExtras.getPartialResultCount() < mTotalPartialCount);
                int requestType = request.getRequestType();

                // Check if we have a callback for this
                if (holder == null) {
                    if (DEBUG) {
                        Log.d(TAG,
                                "holder is null, early return at frame "
                                        + frameNumber);
                    }

                    mFrameNumberTracker.updateTracker(frameNumber, /*result*/null, isPartialResult,
                            requestType);

                    return;
                }

                if (isClosed()) {
                    if (DEBUG) {
                        Log.d(TAG,
                                "camera is closed, early return at frame "
                                        + frameNumber);
                    }

                    mFrameNumberTracker.updateTracker(frameNumber, /*result*/null, isPartialResult,
                            requestType);
                    return;
                }


                Runnable resultDispatch = null;

                CaptureResult finalResult;
                // Make a copy of the native metadata before it gets moved to a CaptureResult
                // object.
                final CameraMetadataNative resultCopy;
                if (holder.hasBatchedOutputs()) {
                    resultCopy = new CameraMetadataNative(result);
                } else {
                    resultCopy = null;
                }

                // Either send a partial result or the final capture completed result
                if (isPartialResult) {
                    final CaptureResult resultAsCapture =
                            new CaptureResult(result, request, resultExtras);
                    // Partial result
                    resultDispatch = new Runnable() {
                        @Override
                        public void run() {
                            if (!CameraDeviceImpl.this.isClosed()) {
                                if (holder.hasBatchedOutputs()) {
                                    // Send derived onCaptureProgressed for requests within
                                    // the batch.
                                    for (int i = 0; i < holder.getRequestCount(); i++) {
                                        CameraMetadataNative resultLocal =
                                                new CameraMetadataNative(resultCopy);
                                        CaptureResult resultInBatch = new CaptureResult(
                                                resultLocal, holder.getRequest(i), resultExtras);

                                        holder.getCallback().onCaptureProgressed(
                                            CameraDeviceImpl.this,
                                            holder.getRequest(i),
                                            resultInBatch);
                                    }
                                } else {
                                    holder.getCallback().onCaptureProgressed(
                                        CameraDeviceImpl.this,
                                        request,
                                        resultAsCapture);
                                }
                            }
                        }
                    };
                    finalResult = resultAsCapture;
                } else {
                    List<CaptureResult> partialResults =
                            mFrameNumberTracker.popPartialResults(frameNumber);

                    final long sensorTimestamp =
                            result.get(CaptureResult.SENSOR_TIMESTAMP);
                    final Range<Integer> fpsRange =
                            request.get(CaptureRequest.CONTROL_AE_TARGET_FPS_RANGE);
                    final int subsequenceId = resultExtras.getSubsequenceId();
                    final TotalCaptureResult resultAsCapture = new TotalCaptureResult(result,
                            request, resultExtras, partialResults, holder.getSessionId(),
                            physicalResults);
                    // Final capture result
                    resultDispatch = new Runnable() {
                        @Override
                        public void run() {
                            if (!CameraDeviceImpl.this.isClosed()){
                                if (holder.hasBatchedOutputs()) {
                                    // Send derived onCaptureCompleted for requests within
                                    // the batch.
                                    for (int i = 0; i < holder.getRequestCount(); i++) {
                                        resultCopy.set(CaptureResult.SENSOR_TIMESTAMP,
                                                sensorTimestamp - (subsequenceId - i) *
                                                NANO_PER_SECOND/fpsRange.getUpper());
                                        CameraMetadataNative resultLocal =
                                                new CameraMetadataNative(resultCopy);
                                        // No logical multi-camera support for batched output mode.
                                        TotalCaptureResult resultInBatch = new TotalCaptureResult(
                                            resultLocal, holder.getRequest(i), resultExtras,
                                            partialResults, holder.getSessionId(),
                                            new PhysicalCaptureResultInfo[0]);

                                        holder.getCallback().onCaptureCompleted(
                                            CameraDeviceImpl.this,
                                            holder.getRequest(i),
                                            resultInBatch);
                                    }
                                } else {
                                    holder.getCallback().onCaptureCompleted(
                                        CameraDeviceImpl.this,
                                        request,
                                        resultAsCapture);
                                }
                            }
                        }
                    };
                    finalResult = resultAsCapture;
                }

                final long ident = Binder.clearCallingIdentity();
                try {
                    holder.getExecutor().execute(resultDispatch);
                } finally {
                    Binder.restoreCallingIdentity(ident);
                }

                // Collect the partials for a total result; or mark the frame as totally completed
                mFrameNumberTracker.updateTracker(frameNumber, finalResult, isPartialResult,
                        requestType);

                // Fire onCaptureSequenceCompleted
                if (!isPartialResult) {
                    checkAndFireSequenceComplete();
                }
            }
        }
        ....
}

6.5 onImageAvailable

[->frameworks/base/media/java/android/media/ImageReader.java]

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
 protected ImageReader(int width, int height, int format, int maxImages, long usage) {
      mWidth = width;
      mHeight = height;
      mFormat = format;
      mMaxImages = maxImages;

      if (width < 1 || height < 1) {
          throw new IllegalArgumentException(
              "The image dimensions must be positive");
      }
      if (mMaxImages < 1) {
          throw new IllegalArgumentException(
              "Maximum outstanding image count must be at least 1");
      }

      if (format == ImageFormat.NV21) {
          throw new IllegalArgumentException(
                  "NV21 format is not supported");
      }

      mNumPlanes = ImageUtils.getNumPlanesForFormat(mFormat);

      nativeInit(new WeakReference<>(this), width, height, format, maxImages, usage);

      mSurface = nativeGetSurface();

      mIsReaderValid = true;
      // Estimate the native buffer allocation size and register it so it gets accounted for
      // during GC. Note that this doesn't include the buffers required by the buffer queue
      // itself and the buffers requested by the producer.
      // Only include memory for 1 buffer, since actually accounting for the memory used is
      // complex, and 1 buffer is enough for the VM to treat the ImageReader as being of some
      // size.
      mEstimatedNativeAllocBytes = ImageUtils.getEstimatedNativeAllocBytes(
              width, height, format, /*buffer count*/ 1);
      VMRuntime.getRuntime().registerNativeAllocation(mEstimatedNativeAllocBytes);
  }

/**
   * Called from Native code when an Event happens.
   *
   * This may be called from an arbitrary Binder thread, so access to the ImageReader must be
   * synchronized appropriately.
   */
  private static void postEventFromNative(Object selfRef) {
      @SuppressWarnings("unchecked")
      WeakReference<ImageReader> weakSelf = (WeakReference<ImageReader>)selfRef;
      final ImageReader ir = weakSelf.get();
      if (ir == null) {
          return;
      }

      final Handler handler;
      synchronized (ir.mListenerLock) {
          handler = ir.mListenerHandler;
      }
      if (handler != null) {
          handler.sendEmptyMessage(0);
      }
  }
  
   public void setOnImageAvailableListener(OnImageAvailableListener listener, Handler handler) {
      synchronized (mListenerLock) {
          if (listener != null) {
              Looper looper = handler != null ? handler.getLooper() : Looper.myLooper();
              if (looper == null) {
                  throw new IllegalArgumentException(
                          "handler is null but the current thread is not a looper");
              }
              if (mListenerHandler == null || mListenerHandler.getLooper() != looper) {
                  mListenerHandler = new ListenerHandler(looper);
              }
              mListener = listener;
          } else {
              mListener = null;
              mListenerHandler = null;
          }
      }
  }
  
   private final class ListenerHandler extends Handler {
      public ListenerHandler(Looper looper) {
          super(looper, null, true /*async*/);
      }

      @Override
      public void handleMessage(Message msg) {
          OnImageAvailableListener listener;
          synchronized (mListenerLock) {
              listener = mListener;
          }

          // It's dangerous to fire onImageAvailable() callback when the ImageReader is being
          // closed, as application could acquire next image in the onImageAvailable() callback.
          boolean isReaderValid = false;
          synchronized (mCloseLock) {
              isReaderValid = mIsReaderValid;
          }
          if (listener != null && isReaderValid) {
              listener.onImageAvailable(ImageReader.this);
          }
      }
  }
  
   private synchronized native Surface nativeGetSurface();

之前已经通过两个回调接口onCaptureProgressed以及onCaptureCompleted方法将meta data上传到了App，一般情况下，图像数据会在他们之后上传，而且这个上传过程并不经过Camera Framework，而是通过BufferQueue来进行的，当Camera Service接收到底层传来的图像数据，便会立即调用processCaptureResult_3_4方法，该方法中会去调用BufferQueue中生产者角色的Surface的queueBuffer方法，将数据入队并通知消费者去消费，而此时的消费者正是应用侧的ImageReader，并经过一层层回调，最终会通过调用ImageReader的onImageAvailable方法，通知ImageReader去将数据取出，并做后期操作。

6.6 小结

该方法最终也会调用到Framework中的submitCaptureRequest方法，接下来边和预览流程大致相同，会去调用Camera Service 中的ICameraDeviceUser的submitRequestList方法传入请求，之后将App实现的回调对象存入CameraDeviceImpl对象中。

七、总结

基于接口与实现相分离的基本设计原则，谷歌通过Camera Api 接口的定义，搭建起了App与相机系统的桥梁，而具体实现便是由Camera Framework来负责完成的。在采用Camera Api v1接口的时期，该部分是通过JNI层来进行java到C++的转换，进而到达native层，而在native层会通过实现CameraClient建立与Camera Service的通讯 ，整个过程比较繁琐，使得整体框架略显繁杂，而随着Camera Api v2的提出，在该层便大量使用AIDL机制，直接在Java层建立与Camera Service的通信，进一步简化了整体框架。接下来我们以几个主要接口为主线，简单梳理下其具体实现。

CameraManager

实现主要在CameraManager.java中，通过CameraManager查询、获取以及打开一个Camera 设备。在该类中还实现了内部类CameraManagerGlobal，该类继承于ICameraServiceListener.Stub，在打开相机设备的时候，在内部会获取到ICameraService远程代理，并且调用ICameraService的addListener方法将自己注册到Camera Service中，一旦Camera Service状态有所变更便会通过其实现的回调方法通知到Camera Manager服务，另外，该类还通过调用ICameraService.connectDevice()方法获取到Camera Service中的CameraDevice远程代理，并且将该代理传入CameraDeviceImpl中，进而与Camera Service建立了连接。

CameraDeviceImpl

该类定义在CameraDeviceImpl.java文件中，继承并实现了CameraDevice接口，代表了一个相机设备，可以完成CameraCaptureSession的创建以及CaptureRequest创建等工作，内部定义了CameraDeviceCallbacks类(该类继承于ICameraDeviceCallbacks.Stub，对应于Camera Service中的 ICameraDeviceCallbacks接口)，用于接收来自Camera Service中的Camera Device的状态回调，并且内部维护着一个Camera Service 的远程ICameraDevice代理，进而可以下发图像请求到Camera Service中。

CameraCaptureSessionImpl

该类定义在CameraCaptureSessionImpl.java文件中，继承并实现了CameraCaptureSession接口，每一个相机设备在一个时间段中，只能创建并存在一个CameraCaptureSession，其中该类包含了两种Session，一种是普通的，适用于一般情况下的会话操作，另一种是用于Reprocess流程的会话操作，该流程主要用于对于现有的图像数据进行再处理的操作。该类维护着来自实例化时传入的Surface列表，这些Surface正是包含了每一个图像请求的数据缓冲区。除了以上这几个接口，还有几个接口是需要App部分进行实现的，用于返回App所需要的对象或者数据：

CameraDevice.StateCallback

被应用侧进行继承并实现，用于在调用CameraManager的openCamera方法时，通过参数的形式传入Framework，在Framework中，一旦CameraDeviceImpl创建成功便通过其中的onOpened方法将其返回给App，如果失败，便会通过其他方法返回给App错误信息。

CameraCaptureSession.StateCallback

被应用侧进行继承并实现，用于在调用CameraDevice的createCaptureSession方法时作为参数传入Framework中，一旦创建成功，Framework便会通过调用该类的onConfigured接口返回一个CameraCaptureSessionImpl的对象，如果失败，Framework会调用其onConfigureFailed方法将错误信息返回至App。

CameraCaptureSession.CaptureCallback

被应用侧进行继承并实现，App通过调用CameraCaptureSessionImpl的setReaptingRequest或者capture方法是作为参数传入Framework，一旦Framework接收到来自CameraService的数据时，便会通过调用这个回调类将数据发送至App中。