For the past year or so I have been working on a new project. northern Lights Is a set of libraries for construction Write a desktop Applications, from which most of the building blocks are taken Shine. I do not yet have a clear date when the first edition of Aurora will be available, but in the meantime I want to talk about something I’ve been playing with over the last few weeks.
Skiing Is a library that serves as the graphics engine for Chrome, Android, Flutter, Firefox and many other popular platforms. It was also chosen by Jetbrains as the graphics engine for Compose Desktop. One of the more interesting parts of Skia is SkSL – Skia’s shading language – enables fast and powerful writing of section shadings. While shading is typically associated with processing complex scenes in CGI video games and effects, in this post I’m going to show you how I use Skia shading to process textured wallpapers for desktop apps.
First, let’s start with some screenshots:
Here we see the top of the sample demo frame under five different Aurora skins (Top Down, Fall, Business, Business Blue Steel, Nebula, Nebula Amethyst). Autumn features a flat color fill, while the other four have a horizontal slope (darker at the edges, lighter at the middle) with a curved arch along the top edge. If you look closely, all five also present something else – a muted texture that extends over the entire colored area.
Let’s take a look at another screenshot:
The top line shows a Praline noise texture, One in shades of gray and one in orange. The bottom row shows a brushed metal texture, one in shades of gray and one in orange.
Let’s take a look at how to create these textures with Skia shadows in Compose Desktop.
First, we start with Shader.makeFractalNoise
Who wraps SkPerlinNoiseShader::MakeFractalNoise
:
// Fractal noise shader
val noiseShader = Shader.makeFractalNoise(
baseFrequencyX = baseFrequency,
baseFrequencyY = baseFrequency,
numOctaves = 1,
seed = 0.0f,
tiles = emptyArray()
)
Next, we have a custom SkSL duotone shading that calculates the lumen (brightness) of each pixel, and uses this dome to map the original color to a point between two given colors (light and dark):
// Duotone shader
val duotoneDesc = """
uniform shader shaderInput;
uniform vec4 colorLight;
uniform vec4 colorDark;
uniform float alpha;
half4 main(vec2 fragcoord) {
vec4 inputColor = shaderInput.eval(fragcoord);
float luma = dot(inputColor.rgb, vec3(0.299, 0.587, 0.114));
vec4 duotone = mix(colorLight, colorDark, luma);
return vec4(duotone.r * alpha, duotone.g * alpha, duotone.b * alpha, alpha);
}
"""
This shading gets four hits. The first is additional shading (which will be the fractal noise we created earlier). The next two are two colors, and the last one is alpha (for applying partial transparency).
We are now creating a database of houses to transfer our colors and alpha to this silhouette:
val duotoneDataBuffer = ByteBuffer.allocate(36).order(ByteOrder.LITTLE_ENDIAN)
// RGBA colorLight
duotoneDataBuffer.putFloat(0, colorLight.red)
duotoneDataBuffer.putFloat(4, colorLight.green)
duotoneDataBuffer.putFloat(8, colorLight.blue)
duotoneDataBuffer.putFloat(12, colorLight.alpha)
// RGBA colorDark
duotoneDataBuffer.putFloat(16, colorDark.red)
duotoneDataBuffer.putFloat(20, colorDark.green)
duotoneDataBuffer.putFloat(24, colorDark.blue)
duotoneDataBuffer.putFloat(28, colorDark.alpha)
// Alpha
duotoneDataBuffer.putFloat(32, alpha)
And create our duet shading with RuntimeEffect.makeForShader
Cover for SkRuntimeEffect::MakeForShader
) F RuntimeEffect.makeShader
Cover for SkRuntimeEffect::makeShader
):
val duotoneEffect = RuntimeEffect.makeForShader(duotoneDesc)
val duotoneShader = duotoneEffect.makeShader(
uniforms = Data.makeFromBytes(duotoneDataBuffer.array()),
children = arrayOf(noiseShader),
localMatrix = null,
isOpaque = false
)
With this shading, we have two options for filling in the background of the Compose element. The first is to wrap Skia’s sound in Compose’s ShaderBrush
And use drawBehind
device:
val brush = ShaderBrush(duotoneShader)
Box(modifier = Modifier.fillMaxSize().drawBehind {
drawRect(
brush = brush, topLeft = Offset(100f, 65f), size = Size(400f, 400f)
)
})
The second option is to create a local Painter
Object, use DrawScope.drawIntoCanvas
Block bypass DrawScope.onDraw
, Get the original fabric with Canvas.nativeCanvas
And call drawPaint
On the original canvas (Skia) directly with the Skia Shader we created:
val shaderPaint = Paint()
shaderPaint.setShader(duotoneShader)
Box(modifier = Modifier.fillMaxSize().paint(painter = object : Painter() {
override val intrinsicSize: Size
get() = Size.Unspecified
override fun DrawScope.onDraw() {
this.drawIntoCanvas {
val nativeCanvas = it.nativeCanvas
nativeCanvas.translate(100f, 65f)
nativeCanvas.clipRect(Rect.makeWH(400f, 400f))
nativeCanvas.drawPaint(shaderPaint)
}
}
}))
What about the brushed metal texture? In Aurora it is created by applying modulated sine / cosine waves on top of Perlin’s noise shading. The relevant section is:
// Brushed metal shader
val brushedMetalDesc = """
uniform shader shaderInput;
half4 main(vec2 fragcoord) {
vec4 inputColor = shaderInput.eval(vec2(0, fragcoord.y));
// Compute the luma at the first pixel in this row
float luma = dot(inputColor.rgb, vec3(0.299, 0.587, 0.114));
// Apply modulation to stretch and shift the texture for the brushed metal look
float modulated = abs(cos((0.004 + 0.02 * luma) * (fragcoord.x + 200) + 0.26 * luma)
* sin((0.06 - 0.25 * luma) * (fragcoord.x + 85) + 0.75 * luma));
// Map 0.0-1.0 range to inverse 0.15-0.3
float modulated2 = 0.3 - modulated / 6.5;
half4 result = half4(modulated2, modulated2, modulated2, 1.0);
return result;
}
"""
val brushedMetalEffect = RuntimeEffect.makeForShader(brushedMetalDesc)
val brushedMetalShader = brushedMetalEffect.makeShader(
uniforms = null,
children = arrayOf(noiseShader),
localMatrix = null,
isOpaque = false
)
Then transfer the blur shadow as input to the dueton eyeshadow:
val duotoneEffect = RuntimeEffect.makeForShader(duotoneDesc)
val duotoneShader = duotoneEffect.makeShader(
uniforms = Data.makeFromBytes(duotoneDataBuffer.array()),
children = arrayOf(brushedMetalShader),
localMatrix = null,
isOpaque = false
)
The full tube for creating the two Aurora-textured shadows is Here, And the processing of the textures is done Here.
What if we want our broadcasts to be dynamic? First let’s see some videos:
The full code can be found for both of these demos Here and Here.
The core setting is the same – use Runtime.makeForShader
To assemble the shading section of SkSL, pass parameters with RuntimeEffect.makeShader
, And then use any of them ShaderBrush
+ drawBehind
or Painter
+ DrawScope.drawIntoCanvas
+ Canvas.nativeCanvas
+ Canvas.drawPaint
. The additional setting involved is around dynamically changing one or more shading attributes based on time (and possibly other parameters) and using Compose’s built-in responsive flow to update the pixels in real time.
First, we define our variables:
val runtimeEffect = RuntimeEffect.makeForShader(sksl)
val shaderPaint = remember { Paint() }
val byteBuffer = remember { ByteBuffer.allocate(4).order(ByteOrder.LITTLE_ENDIAN) }
var timeUniform by remember { mutableStateOf(0.0f) }
var previousNanos by remember { mutableStateOf(0L) }
We then update our shading with the time-based parameter:
val timeBits = byteBuffer.clear().putFloat(timeUniform).array()
val shader = runtimeEffect.makeShader(
uniforms = Data.makeFromBytes(timeBits),
children = null,
localMatrix = null,
isOpaque = false
)
shaderPaint.setShader(shader)
So we have our own drawing logic
val brush = ShaderBrush(shader)
Box(modifier = Modifier.fillMaxSize().drawBehind {
drawRect(
brush = brush, topLeft = Offset(100f, 65f), size = Size(400f, 400f)
)
})
And finally, connection effect Which synchronizes our updates with the clock and updates the time-based parameter:
LaunchedEffect(null) {
while (true) {
withFrameNanos { frameTimeNanos ->
val nanosPassed = frameTimeNanos - previousNanos
val delta = nanosPassed / 100000000f
if (previousNanos > 0.0f) {
timeUniform -= delta
}
previousNanos = frameTimeNanos
}
}
}
Now, in every clock frame we update the timeUniform
Changes, then move the new updated value into shading. Compose recognizes that a variable used in the top-level connection element has changed, reassembles it and redraws the content – basically asking our shadow to redraw the relevant area based on the new value.
Stay tuned for more news on Aurora as it approaches its first official release!
- Multiple texture readings are expensive, and you may want to force such paths to draw the texture to
SkSurface
And read his pixels from anSkImage
. - If your shading does not have to create an accurate and perfect replica of the pixels of the target appearance, consider sacrificing some of the finer visual details for performance. For example, a large horizontal blur that reads 20 pixels on each “side” as part of the twist (41 readings per pixel) can be replaced by double or triple activation of a smaller convolution matrix, or reducing the scale of the original image, applying smaller blur and increasing the result.
- Performance matters because your shading (or shading chain) works on every pixel. It can be a high-resolution display (lots of pixels for processing), a low GPU, a processor-connected tube (without a GPU), or any combination of them.