Basically, you scan a rectangle box of the area on the screen where any part of the triangle you want to fill is located (using the outer coordinates of the triangle)
Amiga does a trick with the blitter by a simple circuit that does some bit manipulation when it copies the bit-plane into the DMA-buffer ready to go to the video memory, but this trick is ... not so good and full of defectives. E.g. it's not able to fill a triangle if one of the borders has even points
I have recently bought "Amiga System Programmer's Guide", it shows this kind of problem and tricks to mitigate it, but I believe the problem needs a radically different approach.
i.e. it needs to think about what "
fill a triangle" means, which well ... for a given triangle of vertexes { V1(x,y),V2(x,y),V3(x,y) }, and a point P(x,y), it's almost like the finding "sort-of-
weights" (what?) that tell us how much of P's X coordinate is made of { V1, V2, V3 }, and also the same for P's Y coordinate:
function( P(x,y), V1(x,y),V2(x,y),V3(x,y) ) ----> { W1, W2, W3 }
Three numbers to tell:
W1: how much of P(x,y)'s coordinates is made of V1(x,y) ?
W2: how much of P(x,y)'s coordinates is made of V2(x,y) ?
W3: how much of P(x,y)'s coordinates is made of V3(x,y) ?
(with "sort-of-
weights" I mean this)
I don't know it can be generalized for a generic polygon, but for a triangle it's like solving a linear system of three equations, whose denominators of the first two equations are the same, whose expression of the third equation is the sum of W1,W2,W2, and whose solution { W1,W2,W2 } has an intriguing property:
if P(x,y) is actually inside of the triangle, then
range( W1 ) is { 0 .. 1 }
range( W2 ) is { 0 .. 1 }
range( W3 ) is { 0 .. 1 }
if P(x,y) is actually outside of the triangle, then at least one of { W1, W2, W3 } will be negative!
This can be used as an easy check to see if a point lies inside or outside of a triangle
All good, and exciting, except ...
1) computing { W1,W2,W2 } require six dot-multiplications(1), and two divisions
2) this must be applied to all the points for the rectangle that contains the triangle
life is complex
(1) e.g. dot(v1, v2) = (v1.x * v2.x + v1.x * v2.y) + (v1.y * v2.x + v1.y * v2.y)