Shulou(Shulou.com)06/01 Report--

This article mainly introduces the relevant knowledge of how to use Java+OpenCV to call the camera to achieve photography function, the content is detailed and easy to understand, the operation is simple and fast, and has a certain reference value. I believe you will gain something after reading this article on how to use Java+OpenCV to call the camera to achieve photography function. Let's take a look.

Environmental preparation

1. Prepare a USB external camera, I use two kinds, one is the ordinary Logitech camera, the other is the binocular camera (used for biopsy in the future)

2.eclipse version 2021-12

3.JDK 11, because we use the plug-in Window Builder forms designer to write swing. In eclipse version 2021-12, to drive the Windows Builder forms designer, we must use JDK11 and +

4. Programming using the Windows10 environment. Of course, we can also use Mac, but if the camera is driven by JAVA under Mac, there is a problem like this: directly you cannot call the camera directly in eclipse, it will report a "This app has crashed because it attempted to access privacy-sensitive data without a usage description" or

OpenCV: not authorized to capture video (status 0), requesting...

OpenCV: can not spin main run loop from other thread, set OPENCV_AVFOUNDATION_SKIP_AUTH=1 to disable authorization request and perform it in your application.

OpenCV: camera failed to properly initialize

Such errors are caused by permission problems in Mac OS, which means that you do not have permission to call some of the devices built into Mac under Mac. If you use XCode to write Swift, then you can solve this problem through info.plist. However, because the java main function is started in eclipse, it is impossible to solve the problem of running Mac peripherals in eclipse under Mac OS. If you are in Mac OS, to run OpenCV Java and drive the camera, you must type the project into an executable jar package and launch it with commands such as java-jar under the command window. When starting up, your Mac OS will prompt you to authorize the command window, please click "Yes" and authorize it with your fingerprint or password, and then run the java-jar opencv application in the command window again, and you can use java to drive the camera under Mac OS. Therefore, this brings great inconvenience to our coding debugging, which is the main reason why we use Windows10 to develop opencv java.

Make the main interface

Our main interface is a Java Swing JFrame application that looks like this.

Introduction to the overall structure

We divide the screen into two areas, and the layout uses 10240768, with a free layout that closes the program by clicking the close button:

SetDefaultCloseOperation (JFrame.EXIT_ON_CLOSE); setBounds (100,100,768); contentPane = new JPanel (); contentPane.setBorder (new EmptyBorder (5,5,5,5)); setContentPane (contentPane); contentPane.setLayout (null)

Upper region

We use a JPanel to group it called cameraGroup, and this JPanel is also a free layout.

JPanel cameraGroup = new JPanel (); cameraGroup.setBounds (10,10,988,580); contentPane.add (cameraGroup); cameraGroup.setLayout (null)

Then, in this cameraGroup, two additional JPanel are placed on the left and right side:

VideoCamera

VideoPreview

The videoCamera is a custom JPanel

Protected static VideoPanel videoCamera = new VideoPanel ()

It is used to show that when the camera is turned on, the images taken from the camera are constantly "brushed" to the JPanel to display, the code is as follows:

Package org.mk.opencv; import java.awt.*;import java.awt.image.BufferedImage;import javax.swing.*; import org.mk.opencv.util.ImageUtils;import org.mk.opencv.util.OpenCVUtil;import org.opencv.core.Mat; public class VideoPanel extends JPanel {private Image image; public void setImageWithMat (Mat mat) {image = OpenCVUtil.matToBufferedImage (mat); this.repaint ();} public void SetImageWithImg (Image img) {image = img } public Mat getMatFromImage () {Mat faceMat = new Mat (); BufferedImage bi = ImageUtils.toBufferedImage (image); faceMat = OpenCVUtil.bufferedImageToMat (bi); return faceMat;} @ Override protected void paintComponent (Graphics g) {super.paintComponent (g); if (image! = null) g.drawImage (image, 0,0, image.getWidth (null), image.getHeight (null), this) } public static VideoPanel show (String title, int width, int height, int open) {JFrame frame = new JFrame (title); if (open = = 0) {frame.setDefaultCloseOperation (JFrame.EXIT_ON_CLOSE);} else {frame.setDefaultCloseOperation (JFrame.DISPOSE_ON_CLOSE);} frame.setSize (width, height); frame.setBounds (0,0, width, height) VideoPanel videoPanel = new VideoPanel (); videoPanel.setSize (width, height); frame.setContentPane (videoPanel); frame.setVisible (true); return videoPanel;}}

Lower region

We put a buttonGroup in the lower area. This buttonGroup uses a "net bag layout" with three buttons on it.

JPanel buttonGroup = new JPanel (); buttonGroup.setBounds (65,610,710,35); contentPane.add (buttonGroup); buttonGroup.setLayout (new GridLayout (1,0,0,0))

Today we are going to implement the functions in photoButton.

After that, let's go to the core code to explain the layout.

Explanation of core code and knowledge points

(the final meeting will be full code)

How to display camera Image in JPanel

JPanel is a component that is usually embedded in JFrame's contentPanel (this is a container that comes with JFrame generated by a graphical designer to "hold" other components).

ContentPane can be thought of as a container. It usually has the following relationship:

JFrame (our main class)-> contentPane- > our own top half JPanel- > videoCamera (JPanel).

There is a method in Java Swing called the repaint () method. Once this method is called, the

Protected void paintComponent (Graphics g)

Will automatically be called one after another.

So, we customized a JPanel called VideoPanel, and then we overridden the paintComponent method in it

@ Override protected void paintComponent (Graphics g) {super.paintComponent (g); if (image! = null) g.drawImage (image, 0,0, image.getWidth (null), image.getHeight (null), this);}

In this way, in our main class "FaceRecognize", after we get the image through the camera and pass the image through the "setImageWithMat" method set in VideoPanel, we immediately call FaceRecognize's own repaint method, and then the "parent" event is transmitted all the way down, and the child components are "refreshed" step by step-the paintComponent of the child components will be triggered again.

The process of the camera getting the image and displaying it in the videoCamera area is as follows:

Keep reading Mat objects through the camera in the FaceRecognize class.

Set the Mat object into the VideoPanel

Constantly calling the repaint method in FaceRecognize forces the VideoPanel to "refresh" the content captured by the camera.

Sleep (50 milliseconds) each time it is displayed

In order to achieve a good refresh, continuous display effect, you can put the above methods in a "single thread".

OpenCV calls camera

OpenCV uses the following class to drive the camera.

Private static VideoCapture capture = new VideoCapture ()

Then turn on the camera and read it as follows

Capture.open (0); Scalar color = new Scalar (0,255,0); MatOfRect faces = new MatOfRect (); if (capture.isOpened ()) {logger.info ("> video camera in working"); Mat faceMat = new Mat (); while (true) {capture.read (faceMat); if (! faceMat.empty ()) {faceCascade.detectMultiScale (faceMat, faces); Rect [] facesArray = faces.toArray () If (facesArray.length > = 1) {for (int I = 0; I)

< facesArray.length; i++) { Imgproc.rectangle(faceMat, facesArray[i].tl(), facesArray[i].br(), color, 2); videoPanel.setImageWithMat(faceMat); frame.repaint(); } } } else { logger.info(">

> not found anyinput "); break;} Thread.sleep (80);}}

Through the above code, we can see the 4 steps I described above.

Capture.open (0) means to read the first camera currently connected to your computer. If you run this sentence on mac, some mac have built-in cameras, so this code will drive the default built-in camera of mac.

If (capture.isOpened ()), there must be. Many online tutorials skip this step of detection, resulting in the camera not producing content. In fact, we finally know that the camera driver is wrong or broken, not a code problem. In the end, it takes too much time to debug. In fact, the result is to change the camera.

While (true) is followed by capture.read (faceMat), which means constantly reading the contents of the camera and reading the contents of the camera into a Mat object.

As mentioned earlier, in order to make the process more "smooth" and "silky", I put the process into a single thread and let it run separately without blocking the main interface of Java Swing. At the same time, use the "green" box to "frame" the human face in the picture. To do this, I made a function as follows:

Public void invokeCamera (JFrame frame, VideoPanel videoPanel) {new Thread () {public void run () {CascadeClassifier faceCascade = new CascadeClassifier (); faceCascade.load (cascadeFileFullPath); try {capture.open (0); Scalar color = new Scalar (0,255,0) MatOfRect faces = new MatOfRect (); / / Mat faceFrames = new Mat (); if (capture.isOpened ()) {logger.info ("> video camera in working"); Mat faceMat = new Mat () While (true) {capture.read (faceMat); if (! faceMat.empty ()) {faceCascade.detectMultiScale (faceMat, faces); Rect [] facesArray = faces.toArray () If (facesArray.length > = 1) {for (int I = 0; I)

< facesArray.length; i++) { Imgproc.rectangle(faceMat, facesArray[i].tl(), facesArray[i].br(), color, 2); videoPanel.setImageWithMat(faceMat); frame.repaint(); // videoPanel.repaint(); } } } else { logger.info(">

> not found anyinput "); break;} Thread.sleep (80);} catch (Exception e) {logger.error (" invoke camera error: "+ e.getMessage (), e) Start ();}

Together with our main method, this is how it works:

Public static void main (String [] args) {FaceRecognize frame = new FaceRecognize (); frame.setVisible (true); frame.invokeCamera (frame, videoCamera);} use camera to take pictures

In this chapter, we recognize that this is "a face" in the introduction to OpenCV Java, which is to output a Mat to a jpg file.

In this chapter, we will do a few things in order to get better results:

Proportionally shrink the Mat object obtained by the camera to "videoPreview"

Output the current Mat of the camera to an external file

The above process is also applied to a single thread to not block the display interface of the main class.

Scale the picture proportionally

Located in the ImageUtils class, it gets a Mat and then turns it into a java.awt.Image object

Then use the AffineTransformOp in Image to scale proportionally according to the ratio (image original scale) based on the specified size (width: 165,200). Then convert Image to BufferedImage

Then transfer the BufferedImage back to the Mat to the FaceRecognize main class for the "display" of VideoPanel to display to our preview area, and the preview area is actually declared by the class VideoPanel.

For this reason, we do event programming for photoButton

JButton photoButton = new JButton ("Take Photo"); photoButton.addActionListener (new ActionListener () {public void actionPerformed (ActionEvent e) {logger.info ("> take photo performed"); StringBuffer photoPathStr = new StringBuffer (); photoPathStr.append (photoPath) Try {if (capture.isOpened ()) {Mat myFace = new Mat (); while (true) {capture.read (myFace) If (! myFace.empty ()) {Image previewImg = ImageUtils.scale2 (myFace, 165,200, true); / / proportional scaling TakePhotoProcess takePhoto = new TakePhotoProcess (photoPath.toString (), myFace); takePhoto.start () / / Photo Writing disk videoPreview.SetImageWithImg (previewImg); / / display scaled photos videoPreview.repaint () in the preview interface; / / Let the preview interface render break again } catch (Exception ex) {logger.error ("> take photo error:" + ex.getMessage (), ex);})

TakePhotoProcess is a single thread with the following code:

Package org.mk.opencv.sample; import org.apache.log4j.Logger;import org.opencv.core.Mat;import org.opencv.core.Scalar;import org.opencv.imgcodecs.Imgcodecs; public class TakePhotoProcess extends Thread {private static Logger logger = Logger.getLogger (TakePhotoProcess.class); private String imgPath; private Mat faceMat; private final static Scalar color = new Scalar (0,0,255); public TakePhotoProcess (String imgPath, Mat faceMat) {this.imgPath = imgPath; this.faceMat = faceMat } public void run () {try {long currentTime = System.currentTimeMillis (); StringBuffer samplePath = new StringBuffer (); samplePath.append (imgPath) .append (currentTime) .append (".jpg"); Imgcodecs.imwrite (samplePath.toString (), faceMat); logger.info ("> write image into- >" + samplePath.toString ()) } catch (Exception e) {logger.error (e.getMessage (), e);}

The other two buttons "trainButton" and "identifyButton" are left for the next two chapters. Let's take it one step at a time, so that we can lay a solid foundation.

Finally, the FaceRecognize is run, and then click photoButton. The effect is as follows:

Complete code OpenCVUtil.javapackage org.mk.opencv.util; import java.awt.Image;import java.awt.image.BufferedImage;import java.awt.image.DataBufferByte;import java.util.ArrayList;import java.util.LinkedList;import java.util.List;import java.io.File; import org.apache.log4j.Logger;import org.opencv.core.CvType;import org.opencv.core.Mat; public class OpenCVUtil {private static Logger logger = Logger.getLogger (OpenCVUtil.class) Public static Image matToImage (Mat matrix) {int type = BufferedImage.TYPE_BYTE_GRAY; if (matrix.channels () > 1) {type = BufferedImage.TYPE_3BYTE_BGR;} int bufferSize = matrix.channels () * matrix.cols () * matrix.rows (); byte [] buffer = new byte [bufferSize]; matrix.get (0,0, buffer) / / get all pixels BufferedImage image = new BufferedImage (matrix.cols (), matrix.rows (), type); final byte [] targetPixels = ((DataBufferByte) image.getRaster (). GetDataBuffer ()) .getData (); System.arraycopy (buffer, 0, targetPixels, 0, buffer.length); return image;} public static List getFilesFromFolder (String folderPath) {List fileList = new ArrayList () File f = new File (folderPath); if (f.isDirectory ()) {File [] files = f.listFiles (); for (File singleFile: files) {fileList.add (singleFile.getPath ());}} return fileList;} public static String randomFileName () {StringBuffer fn = new StringBuffer () Fn.append (System.currentTimeMillis ()) .append ((int) (System.currentTimeMillis ()% (10000-1) + 1)) .append (".jpg"); return fn.toString ();} public static List getPicFromFolder (String rootPath) {List fList = new ArrayList (); int fileNum = 0, folderNum = 0; File file = new File (rootPath) If (file.exists ()) {LinkedList list = new LinkedList (); File [] files = file.listFiles (); for (File file2: files) {if (file2.isDirectory ()) {/ / logger.info ("> folder:" + file2.getAbsolutePath ()); list.add (file2) FolderNum++;} else {/ / logger.info ("> file:" + file2.getAbsolutePath ()); FileBean f = new FileBean (); String fileName = file2.getName (); String suffix = fileName.substring (fileName.lastIndexOf (".") + 1) File fParent = new File (file2.getParent ()); String parentFolderName = fParent.getName (); f.setFileFullPath (file2.getAbsolutePath ()); f.setFileType (suffix); f.setFolderName (parentFolderName); fList.add (f); fileNum++ }} File temp_file; while (! list.isEmpty ()) {temp_file = list.removeFirst (); files = temp_file.listFiles () For (File file2: files) {if (file2.isDirectory ()) {/ / System.out.println ("folder:" + file2.getAbsolutePath ()); list.add (file2); folderNum++ } else {/ / logger.info ("> file: + file2.getAbsolutePath ()); FileBean f = new FileBean (); String fileName = file2.getName (); String suffix = fileName.substring (fileName.lastIndexOf (". ") + 1) File fParent = new File (file2.getParent ()); String parentFolderName = fParent.getName (); f.setFileFullPath (file2.getAbsolutePath ()); f.setFileType (suffix); f.setFolderName (parentFolderName); fList.add (f) FileNum++;} else {logger.info ("> file does not exist!");} / / logger.info ("> folder total:" + folderNum + ", file total:" + fileNum); return fList } public static BufferedImage matToBufferedImage (Mat matrix) {int cols = matrix.cols (); int rows = matrix.rows (); int elemSize = (int) matrix.elemSize (); byte [] data = new byte [cols * rows * elemSize]; int type; matrix.get (0,0, data) Switch (matrix.channels ()) {case 1: type = BufferedImage.TYPE_BYTE_GRAY; break; case 3: type = BufferedImage.TYPE_3BYTE_BGR; / / bgr to rgb byte b; for (int I = 0; I

< data.length; i = i + 3) { b = data[i]; data[i] = data[i + 2]; data[i + 2] = b; } break; default: return null; } BufferedImage image2 = new BufferedImage(cols, rows, type); image2.getRaster().setDataElements(0, 0, cols, rows, data); return image2; } public static Mat bufferedImageToMat(BufferedImage bi) { Mat mat = new Mat(bi.getHeight(), bi.getWidth(), CvType.CV_8UC3); byte[] data = ((DataBufferByte) bi.getRaster().getDataBuffer()).getData(); mat.put(0, 0, data); return mat; }}ImageUtils.javapackage org.mk.opencv.util; import java.awt.Color;import java.awt.Graphics;import java.awt.Graphics2D;import java.awt.GraphicsConfiguration;import java.awt.GraphicsDevice;import java.awt.GraphicsEnvironment;import java.awt.HeadlessException;import java.awt.Image;import java.awt.Transparency;import java.awt.geom.AffineTransform;import java.awt.image.AffineTransformOp;import java.awt.image.BufferedImage;import java.io.File;import java.io.IOException; import javax.imageio.ImageIO;import javax.swing.ImageIcon; import org.opencv.core.Mat; public class ImageUtils { /** * 几种常见的图片格式 */ public static String IMAGE_TYPE_GIF = "gif";// 图形交换格式 public static String IMAGE_TYPE_JPG = "jpg";// 联合照片专家组 public static String IMAGE_TYPE_JPEG = "jpeg";// 联合照片专家组 public static String IMAGE_TYPE_BMP = "bmp";// 英文Bitmap（位图）的简写，它是Windows操作系统中的标准图像文件格式 public static String IMAGE_TYPE_PNG = "png";// 可移植网络图形 public static String IMAGE_TYPE_PSD = "psd";// Photoshop的专用格式Photoshop /** * 缩放图像（按高度和宽度缩放） * * @param srcImageFile 源图像文件地址 * @param result 缩放后的图像地址 * @param height 缩放后的高度 * @param width 缩放后的宽度 * @param bb 比例不对时是否需要补白：true为补白; false为不补白; */ public final synchronized static Image scale2(Mat mat, int height, int width, boolean bb) throws Exception { // boolean flg = false; Image itemp = null; try { double ratio = 0.0; // 缩放比例 // File f = new File(srcImageFile); // BufferedImage bi = ImageIO.read(f); BufferedImage bi = OpenCVUtil.matToBufferedImage(mat); itemp = bi.getScaledInstance(width, height, bi.SCALE_SMOOTH); // 计算比例 // if ((bi.getHeight() >

< facesArray.length; i++) { Imgproc.rectangle(faceMat, facesArray[i].tl(), facesArray[i].br(), color, 2); videoPanel.setImageWithMat(faceMat); frame.repaint(); // videoPanel.repaint(); } } } else { logger.info(">

> not found anyinput "); break;} Thread.sleep (80);} catch (Exception e) {logger.error (" invoke camera error: "+ e.getMessage (), e) }. Start ();} / * Create the frame. * / public FaceRecognize () {setDefaultCloseOperation (JFrame.EXIT_ON_CLOSE); setBounds (100,100,768); contentPane = new JPanel (); contentPane.setBorder (new EmptyBorder (5,5,5,5)); setContentPane (contentPane); contentPane.setLayout (null); JPanel cameraGroup = new JPanel (); cameraGroup.setBounds (10,10,988,580); contentPane.add (cameraGroup) CameraGroup.setLayout (null); JLabel videoDescriptionLabel = new JLabel ("Video"); videoDescriptionLabel.setHorizontalAlignment (SwingConstants.CENTER); videoDescriptionLabel.setBounds (0,10,804,23); cameraGroup.add (videoDescriptionLabel); videoCamera.setBorder (new BevelBorder (BevelBorder.LOWERED, null, null)); videoCamera.setBounds (10,43,794,527); cameraGroup.add (videoCamera) / / JPanel videoPreview = new JPanel (); VideoPanel videoPreview = new VideoPanel (); videoPreview.setBorder (new BevelBorder (BevelBorder.LOWERED, null, null); videoPreview.setBounds (807,359,171,211); cameraGroup.add (videoPreview); JLabel lblNewLabel = new JLabel ("Preview"); lblNewLabel.setHorizontalAlignment (SwingConstants.CENTER); lblNewLabel.setBounds (807307171,42) CameraGroup.add (lblNewLabel); JPanel buttonGroup = new JPanel (); buttonGroup.setBounds (65,610,710,35); contentPane.add (buttonGroup); buttonGroup.setLayout (new GridLayout (1,0,0,0)); JButton photoButton = new JButton ("Take Photo") PhotoButton.addActionListener (new ActionListener () {public void actionPerformed (ActionEvent e) {logger.info ("> take photo performed"); StringBuffer photoPathStr = new StringBuffer (); photoPathStr.append (photoPath); try {if (capture.isOpened ()) {Mat myFace = new Mat ()) While (true) {capture.read (myFace); if (! myFace.empty ()) {Image previewImg = ImageUtils.scale2 (myFace, 165,200, true) / / TakePhotoProcess takePhoto = new TakePhotoProcess (photoPath.toString (), myFace); takePhoto.start (); / / videoPreview.SetImageWithImg (previewImg); / / display scaled photos videoPreview.repaint () in the preview interface. / / Let the preview interface render break;}} catch (Exception ex) {logger.error ("> take photo error:" + ex.getMessage (), ex) }); buttonGroup.add (photoButton); JButton trainButton = new JButton ("Train"); buttonGroup.add (trainButton); JButton identifyButton = new JButton ("Identify"); buttonGroup.add (identifyButton);}} this is the end of the article on "how to use Java+OpenCV to call the camera to take pictures". Thank you for reading! I believe you all have a certain understanding of "how to use Java+OpenCV to call the camera to achieve photo function". If you want to learn more, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.