For a long time, I am very curious about Three.js, WebGL. But, it always feel overwhelming to me. I have tried WebGL Fundamentals before,but somehow I couldn’t persist because there were always some areas I didn’t understand, mainly linear algebra.

About a year ago, I see some very impressive demos on Twitter.

This was a C4D project. Later, the author of R3F posted a demo that blew my mind.

I began to dig into the source code and soon realized I had to learn the fundamentals of WebGL. So I decided to give it another try. I followed the WebGL Fundamentals articles and typed all the code. After reading “2D translation, rotation, scale, matrix math,” I decided to pick up linear algebra. Fortunately, there are tons of learning materials about linear algebra, such as textbooks.math.gatech.edu and immersivemath. I often referred to another website when I felt confused. After about a month, I thought I was ready to continue my WebGL fundamentals journey, so I started to read the R3F source code. After lots of ChatGPT-ing and googling, I decided to write an article. Maybe this will give me a better understanding.

Teaching is the highest form of understanding

The most simple case 2d refraction

Consider a 2D plane and a background. What will it look like when viewed through the 2D plane? The first thing that comes to mind is refraction. Light bends when traveling through different materials.

This is a great article written by Maxime. You can play this demo by clicking the link in the description. There are only a glass and an object. So, how do we solve this? Simple!

• render the scene without the glass and save the render data
• pass the image data to glass mesh shader and calculate the final color output
• render the glass

Here is a code snippet taken from Maxime.

const { gl, scene, camera } = state;
//  render the scene without the *glass*
mesh.current.visible = false;
gl.setRenderTarget(mainRenderTarget);
// and save the render data
gl.render(scene, camera);
 
// pass the image data to *glass* mesh shader and calculate the final color output
mesh.current.material.uniforms.uTexture.value = mainRenderTarget.texture;
 
gl.setRenderTarget(null);
// render the *glass*
mesh.current.visible = true;

Steps 1 and 3 are easy to understand. Step 2 is where the magic happens. After applying the refraction effect in shaders, you can see that we have a refraction effect.

The next step is to make this more realistic. What we could do includes:

dispersion

uniform float uIorR;
uniform float uIorG;
uniform float uIorB;
//...
 
void main() {
  float iorRatioRed = 1.0/uIorR;
  float iorRatioGreen = 1.0/uIorG;
  float iorRatioBlue = 1.0/uIorB;
 
  vec3 color = vec3(1.0);
 
  vec2 uv = gl_FragCoord.xy / winResolution.xy;
  vec3 normal = worldNormal;
 
  vec3 refractVecR = refract(eyeVector, normal, iorRatioRed);
  vec3 refractVecG = refract(eyeVector, normal, iorRatioGreen);
  vec3 refractVecB = refract(eyeVector, normal, iorRatioBlue);
 
  float R = texture2D(uTexture, uv + refractVecR.xy).r;
  float G = texture2D(uTexture, uv + refractVecG.xy).g;
  float B = texture2D(uTexture, uv + refractVecB.xy).b;
 
  color.r = R;
  color.g = G;
  color.b = B;
 
  gl_FragColor = vec4(color, 1.0);
}

multiple sampling

uniform float uRefractPower;
uniform float uChromaticAberration;
 
// ...
 
vec3 color = vec3(0.0);
 
for ( int i = 0; i < LOOP; i ++ ) {
  float slide = float(i) / float(LOOP) * 0.1;
 
  vec3 refractVecR = refract(eyeVector, normal, iorRatioRed);
  vec3 refractVecG = refract(eyeVector, normal, iorRatioGreen);
  vec3 refractVecB = refract(eyeVector, normal, iorRatioBlue);
 
  color.r += texture2D(uTexture, uv + refractVecR.xy * (uRefractPower + slide * 1.0) * uChromaticAberration).r;
  color.g += texture2D(uTexture, uv + refractVecG.xy * (uRefractPower + slide * 2.0) * uChromaticAberration).g;
  color.b += texture2D(uTexture, uv + refractVecB.xy * (uRefractPower + slide * 3.0) * uChromaticAberration).b;
}
 
// Divide by the number of layers to normalize colors (rgb values can be worth up to the value of LOOP)
color /= float( LOOP );
 
//...

3d world

backface render

reflection,Specular & Diffuse

There is a model called the Blinn–Phong reflection model. You can check out 入门Shading，详解Blinn-Phong和Phong光照模型 or Advanced Lighting for a better understanding.

Fresnel effect

FRESNEL EFFECT: this effect establishes that the amount of reflected light rays that we perceive depends on the angle at which we observe the material. A material reflects less light when we look directly at it (at an angle of 0 °, called “fresnel zero”, or F0), and reflects more light at grazing angles. For example, when we enter a lake and look down, we can see the bottom clearly (because looking directly the water reflects very little light). If we look at the horizon we realize that the water reflects much more light, almost like a mirror. The minimum reflectivity value of a material, the F0, is the value that we will use on some of our maps. The PBR shader automatically transitions between F0 and maximum reflectivity according to the angle. (source: everything-about-pbr-textures-and-a-little-more-part-1)

conclusion