Shulou(Shulou.com)06/03 Report--

This article is about how three.js uses gpu to select objects and calculate the location of intersections. The editor thinks it is very practical, so share it with you as a reference and follow the editor to have a look.

Ray casting method

It is very easy to select an object using three.js 's built-in ray casting device (Raycaster). The code is as follows:

Var raycaster = new THREE.Raycaster (); var mouse = new THREE.Vector2 (); function onMouseMove (event) {/ / calculate the device coordinates of the mouse location / / all three coordinate components are-1 to 1 mouse.x = event.clientX / window.innerWidth * 2-1; mouse.y =-(event.clientY / window.innerHeight) * 2 + 1 } function pick () {/ / use the camera and mouse position to update the selected light raycaster.setFromCamera (mouse, camera); / / calculate the object var intersects = raycaster.intersectObjects (scene.children) that intersects the selected light;}

It is realized by using bounding box filtering to calculate whether the projected light intersects each triangle.

However, when the model is very large, such as 400000 faces, it will be very slow to select objects and calculate the location of collision points by traversing, and the user experience is not good.

But using gpu to select objects does not have this problem. No matter how large the scene and model are, the position of the object and intersection of the mouse point can be obtained in one frame.

Use GPU to select objects

The implementation is simple:

1. Create a selected material to replace the material of each model in the scene with a different color.

two。 Read the pixel color of the mouse position and judge the object of the mouse position according to the color.

Specific implementation code:

1. Create a selected material, traverse the scene, and replace each model in the scene with a different color.

Let maxHexColor = 1 if / change the selected material scene.traverseVisible (n = > {return; (! (n instanceof THREE.Mesh)) {return;} n.oldMaterial = n. Material; if (n.pickMaterial) {/ / has created the selected material n.material = n. Pickled material; return } let material = new THREE.ShaderMaterial ({vertexShader: PickVertexShader, fragmentShader: PickFragmentShader, uniforms: {pickColor: {value: new THREE.Color (maxHexColor)}); n.pickColor = maxHexColor; maxHexColor++; n.material = n.pickMaterial = material;}) PickVertexShader:void main () {gl_Position = projectionMatrix * modelViewMatrix * vec4 (position, 1.0);} PickFragmentShader:uniform vec3 pickColor;void main () {gl_FragColor = vec4 (pickColor, 1.0);}

two。 Draw the scene on WebGLRenderTarget, read the color of the mouse position, and judge the selected object.

Let renderTarget = new THREE.WebGLRenderTarget (width, height); let pixel = new Uint8Array (4); / / draw and read pixels renderer.setRenderTarget (renderTarget); renderer.clear (); renderer.render (scene, camera); renderer.readRenderTargetPixels (renderTarget, offsetX, height-offsetY, 1, 1, pixel); / / read mouse position color / / restore the original material and get the selected object const currentColor = pixel [0] * 0xffff + pixel [1] * 0xff + pixel [2] Let selected = null;scene.traverseVisible (n = > {if (! (n instanceof THREE.Mesh)) {return;} if (n.pickMaterial & & n.pickColor = currentColor) {/ / same color selected = n; / / object where the mouse is located} if (n.oldMaterial) {n.material = n.oldmaterial; delete n.oldMaterial })

Description: offsetX and offsetY are mouse positions, and height is canvas height. The meaning of the readRenderTargetPixels line is to select the color of pixels with a width of 1 and a height of 1 with the mouse location (offsetX, height-offsetY).

Pixel is Uint8Array (4), which stores four channels of rgba color, each with a range of values of 0,255.

Complete implementation code: https://gitee.com/tengge1/ShadowEditor/blob/master/ShadowEditor.Web/src/event/GPUPickEvent.js

Using GPU to get the location of intersection

The implementation is also simple:

1. Create a depth shader material to render the scene depth onto WebGLRenderTarget.

two。 Calculate the depth of the mouse position, and calculate the intersection location according to the mouse position and depth.

Specific implementation code:

1. Create a depth shader material, encode the depth information in a certain way, and render it to WebGLRenderTarget.

Depth material:

Const depthMaterial = new THREE.ShaderMaterial ({vertexShader: DepthVertexShader, fragmentShader: DepthFragmentShader, uniforms: {far: {value: camera.far}); DepthVertexShader:precision highp float;uniform float far;varying float depth;void main () {gl_Position = projectionMatrix * modelViewMatrix * vec4 (position, 1.0); depth = gl_Position.z / far;} DepthFragmentShader:precision highp float;varying float depth Void main () {float hex = abs (depth) * 16777215.0; / / 0xffffff float r = floor (hex / 65535.0); float g = floor ((hex-r * 65535.0) / 255.0); float b = floor (hex-r * 65535.0-g * 255.0); float a = sign (depth) > = 0? 1.0: 0; / / depth greater than or equal to 0 Less than 0, 0.0. Gl_FragColor = vec4 (r / 255.0, g / 255.0, b / 255.0, a);}

Important note:

A. gl_Position.z is the depth in camera space and is linear, ranging from cameraNear to cameraFar. You can use the shader varying variable for interpolation directly.

B. the reason for gl_Position.z / far is to convert the value to the range of 0 to 1 so that it can be output as a color.

c. Can not use the depth in the screen space, after perspective projection, the depth becomes-1 to 1, most of them are very close to 1 (more than 0.9), not linear, almost unchanged, the color of the output is almost unchanged, very inaccurate.

d. The depth method in the chip element shader: the camera space depth is gl_FragCoord.z and the screen space depth is gl_FragCoord.z / gl_FragCoord.w.

e. The above descriptions are all for perspective projection. Gl_Position.w is 1 in orthographic projection, and the depth of camera space and screen space are the same.

f. In order to output the depth as accurately as possible, the output depth of three components of rgb is adopted. The gl_Position.z/far range is 0: 1, multiplied by 0xffffff, converted to a rgb color value, and r component 1 represents 65535 g component 1 means 255 Magi b component 1 represents 1.

Complete implementation code: https://gitee.com/tengge1/ShadowEditor/blob/master/ShadowEditor.Web/src/event/GPUPickEvent.js

two。 Read the color of the mouse position and restore the read color value to the camera space depth value.

a. The depth after encryption processing is drawn on WebGLRenderTarget. Read color method

Let renderTarget = new THREE.WebGLRenderTarget (width, height); let pixel = new Uint8Array (4); scene.overrideMaterial = this.depthMaterial;renderer.setRenderTarget (renderTarget); renderer.clear (); renderer.render (scene, camera); renderer.readRenderTargetPixels (renderTarget, offsetX, height-offsetY, 1, 1, pixel)

Description: offsetX and offsetY are mouse positions, and height is canvas height. The meaning of the readRenderTargetPixels line is to select the color of pixels with a width of 1 and a height of 1 with the mouse location (offsetX, height-offsetY).

Pixel is Uint8Array (4), which stores four channels of rgba color, each with a range of values of 0,255.

b. Decrypt the camera space depth value after "encryption" to get the correct camera space depth value.

If (pixel [2]! = = 0 | | pixel [1]! = = 0 | | pixel [0]! = = 0) {let hex = (this.pixel [0] * 65535 + this.pixel [1] * 65535 + this.pixel [2]) / 0xfffff; if (this.pixel [3] = 0) {hex =-hex;} cameraDepth =-hex * camera.far / / the depth of the mouse point in the camera coordinate system (note: the depth value in the camera coordinate system is negative)}

3. According to the position of the mouse on the screen and the depth of the camera space, the coordinates in the world coordinate system of the intersection are calculated by interpolation.

Let nearPosition = new THREE.Vector3 (); / / coordinates of the mouse screen position in the camera coordinate system at near let farPosition = new THREE.Vector3 (); / / coordinates of the mouse screen position in the camera coordinate system at far let world = new THREE.Vector3 (); / / World coordinates / / device coordinates const deviceX = this.offsetX / width * 2-1 Const deviceY =-this.offsetY / height * 2 + 1 / / near point nearPosition.set (deviceX, deviceY, 1); / / screen coordinate system: (0,0,1) nearPosition.applyMatrix4 (camera.projectionMatrixInverse); / / camera coordinate system: (0,0,-far) / / far point farPosition.set (deviceX, deviceY,-1); / / screen coordinate system: (0,0,-1) farPosition.applyMatrix4 (camera.projectionMatrixInverse) / / camera coordinate system: (0,0,-near) / / in the camera space, the camera space x and y values are calculated proportionally according to the depth. Const t = (cameraDepth-nearPosition.z) / (farPosition.z-nearPosition.z); / / converts the intersection from the coordinates in the camera space to the world coordinate system. World.set (nearPosition.x + (farPosition.x-nearPosition.x) * t, nearPosition.y + (farPosition.y-nearPosition.y) * t, cameraDepth); world.applyMatrix4 (camera.matrixWorld)

Related application

Use gpu to select objects and calculate the position of intersections, which is often used in situations where very high performance is required. For example:

1. Mouse over the hover effect of the 3D model.

two。 When you add a model, the model previews the effect of putting the model into the scene in real time as the mouse moves.

3. Distance measurement, area measurement and other tools, lines and polygons move on the plane with the mouse, preview the effect in real time, and calculate the length and area.

4. The scene and model are very large, the selection speed of ray casting method is very slow, and the user experience is very poor.

Here is a picture of using gpu to select objects and achieve the mouse hover effect. The red border is the selection effect, and the yellow translucent effect is the mouse hover effect.

You don't understand? Maybe you are not familiar with all kinds of projection operations in three.js. The projection operation formula in three.js is given below.

Projection Operation in three.js

1. ModelViewMatrix = camera.matrixWorldInverse * object.matrixWorld

2. ViewMatrix = camera.matrixWorldInverse

3. ModelMatrix = object.matrixWorld

4. Project = applyMatrix4 (camera.matrixWorldInverse) applyMatrix4 (camera.projectionMatrix)

Unproject = applyMatrix4 (camera.projectionMatrixInverse) .applyMatrix4 (camera.matrixWorld)

6. Gl_Position = projectionMatrix * modelViewMatrix * position

= projectionMatrix * camera.matrixWorldInverse * matrixWorld * position

= projectionMatrix * viewMatrix * modelMatrix * position

Thank you for reading! This is the end of the article on "how three.js uses gpu to select objects and calculate the location of intersection points". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, you can share it out for more people to see!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.