Sunday, September 20, 2015

You can talk “the Talk”, but can you loop “the Loop”?

Humans, parrots, donkeys, flies and pigs have that thing called a "heartbeat". X times per seconds it pumps around our blood. Games do the same more or less, with the only difference that an insane high heartbeat will make you drop dead, while games thrive on it. You should try to get the cyclus done at least 30 times per second.

This article is meant for beginners, explaining the GameLoop and more specifically the role of timers in a game. At the end, a small example of the Engine22 eGameLoop component is given.

Get ready for the Launch!

When programming something, there are generally three ways to execute something:
·         Single shot
·         Event driven
·         Looped

Batch files or scripts are usually good examples of a single-shot program. You start the program, a list of instructions is executed one-by-one (eventually pausing to query the operator to do X yes or no), and then it shuts down. Copying or converting files, downloading data, starting an advanced calculation in the background, unpacking files, printing a document, et cetera.

Event Driven programs are your usual Desktop Application. It starts, initializes stuff, and then waits for user-input. Press key, click a button, swipe the screen, drag & drop items, and so on. A Form-based application made in Delphi or .NET are good examples. Components like a button or listview generate all kinds of events when being pressed, entered, changed, moved-over, and so on. On a deeper layer, the OS (Windows, Linux, …) is registering input and sends your application “messages”, which are then translated to events you can chose to use.

By the way, in industrial types of programs, events can also be “interrupts”. Increment counter if sensorX generates a pulse, engage safety procedure when door-sensor loses signal, et cetera. Just saying input doesn’t have to be a mouse-click or keyboard button. A mouse-click or button-press by the way generates an interrupt in the OS as well, causing a message being sent to your application.

C’mon baby Do the Loop

And then we have the “Looped” kind of program. Grandpa telling the same story over and over and over again, and again. Until you knock his head with a wheelchair maybe. You write code, and then tell to repeat this code until X happens. Where X could be the end-of-your-program, a STOP signal, or whenever this (sub)task is considered completed/aborted. Note it executes the same code again and again, but that doesn’t mean the exact same thing has to happen. A typical scenario is that we check input parameters each “cycle” which may alter the routing (if button1Pressed then … else … ) or behaviour of other code parts.

Industrial applications, often running on PLC’s or Micro-Controllers, are usually doing “the loop” approach. Each cycle, they check input(sensors / touchscreen), run regulators, and send output (relays, pumps, valves, servo’s). It’s the heartbeat that keeps the program going. And if there is no heartbeat (stuck in a loop, error raised, hardware issue), the “Watchdog” will time-out and reset the chip.

And games aren’t much different. Note that applications can combine both events and (multiple!) loops, but again the “Game Loop” is what makes your game drawing, moving, sounding and doing stuff continuously.

You probably know how to make an ordinary loop:

            while not ( flagSTOP ) do begin
                                runTask(  ….  );

There is one little-big problem with this code though; no matter how simple “runTask( … )” is, your CPU will go 100% and everything else freezes. It’s trying to execute “runTask” as much as possible, no pausing, no breathing-space for any other application.

This piece of code has two problems. We have no speed-control (the heartbeat pacemaker going mad). And since it’s executed in the main-thread, this means it will block other main-thread tasks in the same program, such as refreshing the screen or handling button-clicks. As for the speed-control, one seemingly simple solution would be using a Timer component. Delphi, .NET, Qt, Java, they all have Timer components that can run code each X milliseconds. The “idle” time between two cycles is then available for other stuff.

Real-time TV

Problem solved then? Hmmm, not quite. At least, it depends on the accuracy we want. See, your default Timer isn’t very accurate. Why? Because the Windows OS (don’t know for Linux) itself doesn’t have a very accurate timing. Why? Because Windows OS isn’t designed as a “Real-time” operating system? Why? Because Windows doesn’t need to be. Why? Because Windows typically isn’t used on time-critical hardware, such as a vehicle ECU or a packaging machine controller. Why? Ah, drop dead.

Though we all love a good performance, 99,9% of the Windows applications isn’t time-critical. People won’t die if we miss a frame, and we don’t control a packaging machine that screws up if its pneumatic valve is powered 10 milliseconds late. Machinery often requires a real-time system to guarantee the same predictable results over and over again (24/7!). Which is why you shouldn’t use a Windows computer for that. Why? I told you why, because the Windows OS isn’t designed for real-time applications. It does not guarantee that your program will execute the loop again over 872 microseconds. On a typical Windows computer, we have dozens of other programs and background tools running at the same time, which can all claim the CPU or memory, so we simply can’t give any guarantees.

Now a game isn’t time-critical in the sense that you will get hurt if the framerate drops from 60 to 30 in all of a sudden. Although… I’ve seen kids on Youtube that went ape after getting shot in a Death-match game due a computer hick-up… Nevertheless, we want a smooth experience. If your television refresh rate fluctuates between 15 to 100 FPS all the time, you’ll be puking after five minutes. Like in machinery, we want to run our game at a certain pace, and guard that interval.

Say we use a Delpti TTimer and put the interval on 16,66 milliseconds. That means the timer-event is triggered 1000/16,66 = 60 times per second (thus FPS = 60). In theory. We’ll use this event to execute our game-loop. In this loop we could:


Now since we have only 16,66 milliseconds between 2 cycles, that is quite a lot to do in a short time!! Yep, true, but you’ll be amazed what a computer can do. Don’t forget about half of the work is done by another processor-set, the video-card, which is on steroids. The picture above implies everything happens in sequence, but it's common to have some multi-threading going on to execute these sub-tasks in parallel. While the video-card is drawing, you could update audio for example. You’ll also learn that a lot of programs out there are slow as shit because bad programming. If your photo-realistic game can poop 40 frames per second, there is no reason why some silly editing program takes 5 seconds to do a single simple task.

A good engine is a powerful beast, doing thousands, no MILLIONS!, of calculations every second. The key to get this fast is not really doing formula’s in a very optimized way, but avoiding stuff that doesn’t have to be calculated (every cycle). Don’t do extensive searches if you could also do that just once during the loading-phase. Use smart algorithms to to reduce searching lengths. Don’t calculate advanced physics for an object miles away. Don’t animate a 30 legged centipede that is somewhere behind you. Anyway, optimizing is another story.

60 Frames per Second is a nice strive, though I’m already happy if the game just keeps above the 30 FPS line. Above thirty, the human eye thinks animations are smooth, but lower framerates will appear choppy. How many frames you can reach, depends on what you’re doing (how complex & how many entities), and how fast the computer is obviously.

If we’re trying to do more than the computer can handle, it means the framerate will drop and our timer will be executed repeating without any delays between the cycles. This can be temporarily, for example when entering a heavy scene, or if the computer is doing a Virus-Scan on the background (slowdowns aren’t always our fault!). Doctors making rush-hour on the ER, more bloody patients coming in than we can handle. If it’s structural, meaning we never reach our desired framerate and the CPU hitting 100% all the time, we should consider lowering the target framerate (allow less patients), doing less code/optimize (train our doctors), or switch to a more powerful platform (a bigger hospital).

Take a break

At the other hand it may happen we can easily perform our tasks in the given timeframe. If the timer runs at 58,8 FPS, our timeframe is 1000/58,8 = 17 milliseconds. If doing our GameLoop only takes 10 milliseconds, we have 7 more “free time” milliseconds. Which is great, because this allows us to do other stuff, it gives some room to other background applications, and otherwise at least the CPU isn’t running 100%, making a lot of noise all the time.

Here is the tricky part though. After you finished the Game-Loop, you should check how much time that took, and how much take you can rest before taking the next step. A Delphi TTimer does that, but not very accurately, because non-real-time Windows isn’t generating timer-pulses very precisely on the background. That also causes our beloved “sleep( )” function to be unreliable. Calling “sleep(1)” may actually put your thread in bed for 10 milliseconds or so. So, how to keep a steady framerate then? 

Engine22 “eMainloop” class

There are several High-Precision timer components for Delphi, and I’m sure the same thing is available for .NET or any other language. Engine22 also provides the “eMainloop” class (E22_Util.Mainloop), which is sort of a timer. You set it up by giving a desired target-framerate -which is basically a maximum speed-, and a Callback function. This callback function is called every time the eMainLoop object triggers. So, typically you execute your game (or whatever it is you’re making) stuff there.

                Looper : eMainloop;

                Procedure TForm1.initialize();
self.looper             := eMainloop.create( self.handle );
self.looper. setTargetSpeed( 60 );
self.looper.setCallback( self. gameMainLoop );
self.looper.enabled := true;

                procedure TForm1.gameMainLoop( const deltaTime : eFloat );
                end; // gameMainLoop

So “gameMainLoop” gets called, 60 times per second (hopefully) in this example. The elapsed time between 2 frames, in theory 1000/60 = 16,66 ms is given as an argument you can use. How to? Check the “DeltaTime” part at the bottom of this article:

How it solves the timing issue? By not using the Windows OS timer messages, but using the Windows vWMTick_ASAP signal instead. This one is given mega-fucking-fast. The application is allowed to process messages every time we receive a tick, and we measure the elapsed time using the Windows QueryPerformanceCounter() function. This function returns an ever incrementing tick-counter, which can be converted to milliseconds by dividing it with the clock frequency, which you can get via QueryPerformanceFrequency( ptrFrequency ). If the elapsed time exceeds our targetframerate, we call the given callback, “gameMainLoop” in this case.

Enough for today. For people with some experience, this whole story probably sounds all too obvious. But for a newcomer, it's probably good to understand what's going on. After all, game-code doesn't quite look like a common (Event driven) Desktop program. And since the looping/timer is such an essential thing, you'd better not rely on the standard Timer and dig a bit deeper in order to get control.

Saturday, September 5, 2015

Back to school: 3D Vectors

This (beginners) tutorial is written for Engine22 users, but also if you just want to know more about making games, engines, and 3D graphics. More tutorials will follow for sure, and can be found on (and at the time of writing, the link isn't up yet!). Have fun!

Remember Vectors from physics classes? Don't worry, neither do I. No but seriously, I'm a loser when it comes to math. Some parents say their kid is like a sponge. I'm more like bowling ball, without holes. If I don't apply it immediately, my brain does a toilet flush and the gained knowledge exits the body through farts or nose pickles.

Apply it immediately”... what healthy 16 year old applies Pythagoras or Fresnel lens formula's for his daily problems? Only Newton's gravity law seems to apply sometimes, when falling drunk of your bike. E=MC2 in your face, stupid. So, there we are. In a classroom, thinking about the next part in Half life(1), staring at girls boobs, drawing idiotic doodles on a piece of paper, waiting until you can finally leave that sweaty moldy reeking place. Somewhere in the background, an old teacher with glasses is drawing "arrows" on the chalkboard. Whatever man.

But who would have thought that those arrows are called "Vectors", and that I would use them on a daily base, 17 years later? Just as realistic as using the leapfrog you learned at gymnastics.

As said, I'm just not that good at math so I won't be able to explain all the fine details about the math behind vectors. Then again, not being a professor, I can hopefully teach you a few things in human-language. While I'll put on my glasses and walk to the chalkboard, please stop chewing bubble-gum, stop doodling on your note block, and pay attention class. It's not that hard really, and if you want to make a game, you'd better stop watching your neighbour’s boobs, and focus on the chalkboard. Ahum:


Earth is flat, and coordinates in space are made of 3 components: x,y and z. Don't believe me? Look at your finger. Move it to the left. Now it moved into a negative X direction; your finger's X-coordinate decreased. Move up. Your finger's Y coordinate increased. And how about that Z component then? Simple, just move your finger backwards, away from you. Z, not from Zorro but "depth", coordinate increased. From your perspective at least. In your bedroom door opening, your mom is looking at you now, and thinking what the hell you're doing. From her perspective, that finger moved into different directions, and she is about to call an ambulance.

So, a position, also called "point", can be defined as {x,y,z}. A 3-component struct in programming terms, like this:
// pseudo Engine22 notation
eVec3 = record
       x,y,z : eFloat;
And a point can be in different "spaces". In general, we have "local space" (or "model space") and "world space". The finger example explained the coordinates, relative to your own position. Move it up, and Y increases. If you were the centre of the universe -as your mom always says-, your body would be at {x:0, y:0, z:0}. But you're not. For a Chinese on the other side of the world, your finger is going down instead of up.

That is... if the Earth was the centre of the universe. Which isn't either. In real-life, coordinate systems are always sort of local. It really depends what you take as a centre point, and also what direction is accounted for "forward", "up" and "left". If you hang upside-down like a bat the whole day, "up" might get a different definition.

Games are a bit easier. We just take a random point as the centre. Or well, random... On a GTA map, you may decide the bottom/left corner, at sea level, would be {0,0,0}. For Tower22, the centre of the ground floor is the centre of the "world". Game-world. When going up, Y goes higher. But there also games/3D programs that take Z as "up". It's arbitrary, and it doesn't really matter just as long your 3D models and programming math all accept the same rules.

In a game-context, "local" coordinates usually apply on/within a 3D model. So we make a large outdoor area. And we place assets in them. Cars, light-posts, garbage containers, trees, cats, et cetera. Each object gets a "world coordinate". However, earlier while creating these objects (say we were modelling a car), we didn't know if/where and how these objects are placed within the world. We just pretend the world is gone, and now the centre of the car itself becomes the "centre of the world". So, probably the handbrake would be somewhere near that centre. The front bumper, is placed forward from the centre, thus having a larger Z coordinate. The rear bumper, "behind" us, would get a negative Z coordinate.

Now when we place this car in our world, we can rotate it 180 degrees. In local-car-space, the handbrake is still the centre, and the bumper is still forward at a higher Z coordinate. But in world-coordinates, you'll get a whole different picture.
So, in practice, every object in your world gets an "absolute", world-coordinate. Physics,  movement, placement, and all that kind of stuff will be calculated with world-positions, world-vectors, and world-matrices (more about that later). But sometimes, you also want to do something in local-space. Or convert from local- to world space, or vice-versa. Example. Your NPC has a sniper rifle, and wants to blow off your head. That happens. Your "head" is a sub-part of your "player-body" object. The NPC knows your player world-coordinate. But next, the exact head-position depends on what the player is doing. If he is in "prone" pose, the head will be low at the ground. If he is licking his own balls, his head will be somewhere in the middle of the whole body-object. The body-object, which drives such animations, knows the local head position. In order to make a good shot, blowing of your head while you were walking on hands with your legs around your neck, the local head coordinate needs to be transformed to world space.

Jesus, that sounds hard. Yeah, it kinda is, and I won't explain the math now. But to do so, you'll need vector & matrix transformation calculations. Which are fortunately very common in any game engine, so you don’t have to reinvent the wheel. For now I'll just illustrate what can be done
with points. And I think... that I'm pretty much done. But to finish this story, let's throw some random (Engine22) code examples to get an idea:

                // Misc. Point code
                function makeYourPointPlease( const x,y,z : eFloat ) : eVec3;
                               result.x := x;
                               result.y := y;
                               result.z := z;
                procedure movePointToTheLeft( var pnt : eVec3; const distance : eFloat );
                               pnt.x := pnt.x - distance;
                function distanceBetween2Points( const A, B : eVec3) : eFloat;
                var x,y,z : eFloat;
                          x      := A.x - B.x;
                          y      := A.y - B.y;
                          z      := A.z - B.z;
                         result := Sqrt(x*x+y*y+z*z);



A point is just a... point, somewhere in a space. And you mainly need them for
- A.I. navigation
- Physics (movement, jumping, collision detection, ...)
- Tell video-card WHERE to draw
- Animations

Now to the Vector. In terms of programming, we can notate it the same. In fact, points and vectors are often the same struct or class in engines. In Engine22, a "vector" can be a vector(duh), but also a point or RGB colour. Anyhow, vectors can be visualized as arrows. Yes, that old man with glasses on the chalkboard wasn't fooling after all. A vector defines one or two things:
                - Direction: arrow point to the left, upwards, a bit backwards, ...

                - Force: The vector length. The longer the vector-arrow, the faster it moves into that direction

Note that "normalized" vectors do not define a Force or Strength, only a direction. A normalized
vector is made in such a way that the Force/Strength/Length is always 1. So a vector pointing to the left, would be noted as {x:-1; y:0 z:0}. An upwards vector would be {x:0 y:+1 z:0}. Polygon “Normals” (the direction its facing), is an example of a normalized vector.

The Matrix

Normalized vectors are used a lot for defining directions in drawing, physics, and rays in graphics(shaders). You see, objects usually don't only have a point, but also a certain direction. Looking from helicopter perspective, you could rotate a car 360 degrees. This angle can be encoded into a direction vector. Although you would need 2 more vectors for the full picture, as you could also roll and flip this car, in case it crashes or something. This is where Matrices are used for. A (4x4) matrix defines 5 things:
                turn / roll / pitch  rotation vectors
Basically a matrix is constructed from multiple vectors. In Engine22, every asset has a matrix. When you place a new object into your world, you define its position, you can rotate it, and eventually scale (shrink / enlarge) it. In other words, you'll be adjusting its matrix while moving it around your mouse or keyboard.

This same matrix is fed into the physics system, so it knows where everything is, and also
shaders will get this matrix so they can position your object correctly on the screen (if visible
at all).

And back to Vectors

Matrix-math is quite a bit harder than vector-math, so let's spare that for another time, and continue this tutorial with some practical programming examples that show how these direction vectors can be used. Let's move bitch:

                procedure movePointUpwards( var targetPoint : eVec3; const unitsUp : eFloat );
                               targetPoint.y := targetPoint.y + unitsUp;
                               // Note it would move down if you give a negative number

                procedure movePoint( var targetPoint : eVec3; const forceVector : eVec3 );
                               // Assume the forceVector is NOT normalized,
                               // thus having a length/strength as well
                               targetPoint.x := targetPoint.x + forceVector.x;
                               targetPoint.y := targetPoint.x + forceVector.y;
                               targetPoint.z := targetPoint.x + forceVector.z;
                procedure movePoint( var targetPoint : eVec3; const direction : eVec3; const speed : eFloat );
                               // Direction vector is normalized here, multiply with speed
                               targetPoint.x := targetPoint.x + direction.x * speed;
                               targetPoint.y := targetPoint.x + direction.y * speed;
                               targetPoint.z := targetPoint.x + direction.z * speed;

Units in 3D space

So... if we move a point "20" to the left... then how far did it move exactly? My old physics teacher would outrage when writing a number without its corresponding unit. The mass of this block would be "10" mister. "10 what?! 10 ounce? 10 grams? 10 donkeys?!". "Kilograms mister". "Then say so". And of course, he was right. But still an asshole.

The guys at OpenGL or DirectX didn't listen very well to their teachers though; there is no
pre-defined unit for 3D coordinates. They basically say: "You figure out.". So, if we move
that point "20" to the left, we should decide ourselves if those are inches, meters,
nautical sea miles, or donkeys. In Engine22, "1" = "1 meter". But in another program it could
just as well be millimeters. So, it's important to pick 1 standard for your unit system, and eventually scale up/down imported models that come from a different 3D package. Lightwave also uses meters, but 3D Max centimetres if I'm not mistaking, so an imported model would be 100x bigger if you forget to downsize.

                procedure checkInput( );
                var movementSpeed : eFloat;
                               movementSpeed := 3.0;  // meters
                               // Move our player with the arrows, over the X and Z axis
                               if ( keyboard.leftArrowPressed ) then
                                               movePoint( player.position,  vec3(-movementSpeed, 0,0 ) );
                               if ( keyboard.rightArrowPressed ) then
                                               movePoint( player.position,  vec3(+movementSpeed, 0,0 ) );
                               if ( keyboard.downArrowPressed ) then
                                               movePoint( player.position,  vec3(0,0, -movementSpeed ) );
                               if ( keyboard.upArrowPressed ) then
                                               movePoint( player.position,  vec3(0,0, +movementSpeed ) );

Stop. DeltaTime!

The procedure above has a little problem. Assuming that your game runs at 60 frames per second. And you'll be checking input every cycle. Thus "checkInput" would be called 60 times per second. Since we move "3.0 meters" every time, your player will go in warp-drive; holding the key for 1 second would move him 3.0 * 60 = 180 meters per second! That's a bit too much, don't you think.

Now we could simply reduce the "movementSpeed", to avoid super-human speeds. 0.03 for example would lead to 0.03 x 60 = 1.8 meters per second.

But this still stinks. What if you run your game on slower computer, that can only reach a lousy 20 FPS at times? 0.03 x 20 = 0.6 meters per second. In the worst case, your FPS fluctuates, because you're downloading internet porn at the same time, causing hitches. Or how about this? You'll buy a quantum computer that easily reaches 1000 FPS. Now suddenly your guy moves with 30 meters per second!

Ever tried old DOS games on an emulator? You may have noticed that some games seem to play in fast-forward. That's because they didn't guard their speeds. One trick is to introduce a maximum FPS. Another (better) trick is to calculate "DeltaTime". In Engine22, most "update()" functions will get a "DeltaSecs" argument with them; the elapsed time in seconds between the current and previous cycle. So if we're at a steady 60 FPS, DeltaSecs would be 1/60 = 0.0166667. Now check this:

                procedure checkInput( const deltaSecs : eFloat );
                var movementSpeed : eFloat;
                               movementSpeed := 3.0 * deltaSecs;     // meters PER SECOND
                               // Move our player with the arrows, over the X and Z axis
                               if ( keyboard.leftArrowPressed ) then
                                               movePoint( player.position,  vec3(-movementSpeed, 0,0 ) );
                               if ( keyboard.rightArrowPressed ) then
                                               movePoint( player.position,  vec3(+movementSpeed, 0,0 ) );
                               if ( keyboard.downArrowPressed ) then
                                               movePoint( player.position,  vec3(0,0, -movementSpeed ) );
                               if ( keyboard.upArrowPressed ) then
                                               movePoint( player.position,  vec3(0,0, +movementSpeed ) );

Problem fixed. We multiply our original "3" with the elapsed time, so it gets a small number, 0.05 if the FPS was 60. And how much is 60 times 0.05? Exactly, 3. We moved 3 meters over a second.

Vectors in Engine22

You saw some of the basics. How to set a point, apply some movement, a bit about local versus world-space. There is a lot more you can do with vectors and matrices though. And therefore one of the basic elements each game-engine should have, is a Vector Library. And so does Engine22 of course. You already saw the "eVec3" type a few times above, which stands for "eVector3". That thing is being used all over the place, and has a bunch of handy help functions.

eVec3 is just a record with 3 floats. Note there is also an integer/byte/float32 variant, as well as a eVec4 type that has an extra W component. One of the reasons Engine22 won't work anymore in Delphi7, is because I made thankful use of record operators & functions, which wasn't possible in the old days. The means you can call functions and do math with vectors:

                var v1, v2, vResult : eVec3;
                               vResult := v1 + v2;           // Sums up (v1.x + v2.x, y+y, z+z)
                               vResult := v1 - v2;            // Subtracts (v1.x - v2.x, ...)
                               vResult := v1 * v2;           // Multiplies (not CROSS!)
                               vResult := vec3( 10, -4.2, 90);     // Constructor
                               vResult := v1.norm();    // Get normalized vector
                               string  := v1.toString();
                               vResult.fromString( "10 -4,2 90" );
                               x             := v2 );// dot product
                               distance:= v1.dist( v2 ); // Distance between v1 and v2
                               lookDir := v1.lookat( targetPosition );    // Gets direction towards target
                               length  := v1.len();          // Get vector length/strength
                               if ( v1 = v2 ) then vectorsAreEqual;
                               if ( v1 > v2 ) then v1IsStrongerThanV2; // Compare forces

Another nice feature is that you can use the same vectors for RGB (or eVec4 for RGBA) colors. Instead of typing "vector.x", you can type "vector.r" as well. It does the exact same thing, but makes more sense in terms of writing. Vector colors are often used as input parameters for shaders and graphics. Note that 0 stands for black, and 1 for 255 (white) here, as we use a floating point notation instead of bytes.

And about 3D models, did you know that...

3D models (the stuff you make in 3D Max, Lightwave, Blender, Maya, ...) are basically just arrays of vectors? A polygon is made of (typically 3) "vertices". Where a vertex could have a position, a texture coordinate, a normal-direction, maybe a tangent, maybe a weight? So, writing it down:

                Vertex = record
                               position               : eVec3;
                               textureCoord    : eVec2;               // Only 2 coordinates (U and V)
                               normal                 : eVec3;
                               tangent                               : eVec3;
                               weight                 : eVec3;
                ModelMesh = class
                               vertexCount      : eInt;
                               vertices               : array of Vertex;

It's not exactly how the Engine22 VertexArray classes look like, but it's not far away from that either. When loading a 3D model, you'll be making arrays of vectors. When rendering the model, you'll be pushing those arrays to your videocard. Just keep that in mind.

It's not exactly how the Engine22 VertexArray classes look like, but it's not far away from that either. When loading a 3D model, you'll be making arrays of vectors. When rendering the model, you'll be pushing those arrays to your videocard. Just keep that in mind.

So much stuff you can do with them. The E22_Util.Base.Vector unit is one of the most common included units. And even if you're not a math genious, you'll get comfortable with them quickly.

To conclude, our deadly sniper headshot:

                procedure sniperHeadShot( targetEntity : eES_Entity );
                var         localHeadPos    : eVec3;
                               worldHeadPos : eVec3;
                               localGunPos      : eVec3;
                               worldGunPos    : eVec3;
                        targetDirection : eVec3;
                        targetDistance  : eFloat;
                        bulletPos    : eVec3;
                        traveledDist              : eFloat;
                               // Get absolute head position
                               localHeadPos    := targetEntity.getLocalPoint( HEAD );
                               worldHeadPos  := targetEntity.getMatrix().localPointToAbsolute( localHeadPos );
                               // Get our gun position
                               localGunPos      := sniper.getLocalPoint( GUN_NOZZLE );
                               worldGunPos    := sniper.getMatrix().localPointToAbsolute( localGunPos );
                               // Get the (world) direction and distance between our gun and the head
                               worldGunPos.distAndDirectionTo( worldHeadPos, targetDirection, targetDistance );
                               // Simulate bullet path
                               bulletPos            := worldGunPos; // Start here
                               traveledDist       := 0.0;                   // Traveled distance so far
                               while ( not collided ) and ( traveledDist < targetDistance ) do
                                               // Check if our path is clear
                                               collided := world.rayTrace( bulletPos, targetDirection, bulletSpeedMS * deltaSecs );
                                               // Move forward, with the speed of a bullet
                                               bulletPos := bulletPos + targetDirection * bulletSpeedMS * deltaSecs;
                                               traveledDist := traveledDist + bulletSpeedMS * deltaSecs;
                               end; // while
                               if ( traveledDist >= targetDist ) then begin
                                               // Reached the target

Of course, that's bullocks code, but it does give an impression of how to work with vectors. The vector-related functions shown are in E22 at least. 

Sunday, August 16, 2015

Build a Pyramid boys

While upgrading (or re-writing really) the engine, I thought it would be nice to share more technical details. Tool development, implemented graphics techniques, engine decisions, et cetera. Since the Engine22 code will be public, its probably a good idea to let you know what's going on right? So, here a post. Oh, and if you wonder where this code can be found, we got web-space: . But please allow me to finish some first tutorials first.

I found it difficult to make a start though. Grabbing a random recently added shader or something isn't difficult, but I always prefer to write a story. With a beginning, and end. With climaxes and plot-twists. With a joke, and drama. So where would we begin this story? The development goes so rapidly, techniques tend to change all the time, and especially the "beginning" -the foundation- of an engine is a wide massive block of code and design strategies. As with most larger software packages, there is no single point to start. An engine is made of many individual or interconnected modules. Some modules can be completely isolated, meaning that they don't require code from another module, and can be easily recycled in any project. For example, a math library. Other modules do depend on other modules; they are "build on-top". Like a pyramid.

A graphics module for example will likely depend on OpenGL or DirectX, and maybe some wrapper module in between. It also depends on your design. When making a "Sound Module" for example, we could keep it stand-alone and just limit its functionality to, well, playing sound! And maybe loading libraries, streaming stuff, doing 3D occlusions, reverb effects... But we could also decide to make it on top of a "World" or "Entity Module", so the Sound module is aware of the map geometry and actors that can emit sound. Choosing the latter usually makes the objects within such a Sound Module more logical, closer to your needs. But on the other hand your Sound module gets much harder to use in another project, as it relies on non-sound-related pieces. And this is exactly where engine (or other software) design goes wrong.

Programmers tend to let their modules(class/functions/structs) do too much. "Because its handy". A Physics-Collider object for example, is primarily made to "bounce". Check if there is a collision between A and B. But hey?! If that happens, we could let the same Physics-Collider trigger sounds and impact particles as well. And reduce the object health if the impact was hard enough. All the logical responses to a collision nicely coded at one place... But wait a minute. Now your physics module is doing physics, as well as sound, graphics, AND game-related events (reducing health, eventually killing the object if it fell head-first from a building)? It's like asking the vegetables-man to not only sell vegetables, but also cook and serve them. It's wrong I tell you.

As logical as it sounds, this brings confusion. If one object does a terrible lot, while another isn't, it gets hard to predict who's going to do what. Should we cook our own diner, or is the vegetables man going to it? And what if we don't want him to do that? In the example above, the physics-collider triggers particles, sounds and reduces health. But what if we don't have such a thing as health, and what if we want very different next-gen effects instead of stupid particles? Long story short, the Physics Module should shut up and keep itself to physics. Let some other, higher-in-the-chain module decide what to do. This brings me back to a pyramid design. Now how does E22 look like, more or less?

I'm a bad pyramid builder.

At the lowest levels, we usually find "Platform" related modules that we can't alter. Depending on our platform choice(s), this is what we get. Obviously the fundaments are different when choosing a Gameboy or PS4. If we want multi-platform support, we'd better use 3rd party modules or write our own wrappers that can switch between platform specific modules. Engine22 doesn't really utilize much Windows functions, and the chosen 3rd party libraries (OpenGL, FMOD or OpenAL, Newton, LUA) aren't bound to a specific OS either. Yet it would require some recompilation and IFDEFs at the deeper layers to make it really cross-platform. At this point, there are no plans for non-Windows platforms yet by the way.

The mid-layers of the pyramid form Engine22. Each brick represents a module, or a separate BPL package in Delphi terms. Usually the lower we get, the module either gets more abstract or specializes itself into a single, very specific task. Low level modules don't implement much game-specific rules and boundaries, making them easier to recycle in a different, eventually non-game project. Sound just plays sound. It doesn't know its role in the bigger whole. But as we climb higher, the modules are gearing more towards an actual game. A 3D Horror survival game like Tower22 I should mention, although "Horror Survival" could just as well be "Romantic Puzzle" or "Kiddy Adventure". Yet, the more higher level modules we use, the more restrictions. Engine22 wasn't designed for a Need for Speed racing game, or a Real Time Strategy Command & Conquer game. Not that it's impossible, but you probably need some add-ons to the physics, or in case of a RTS, a different "World" module as these top-down worlds are a different cookie than our roaming (mainly indoor) worlds.

The high levels are either Demo programs, Engine22 tools, such as our Map Editor, or of course your own program. Your code is in charge of which modules to use, what content to load, and what the "game-rules" are. Engine22 provides all kinds of mechanics for AI pathfinding, weapon damage or doing mini-puzzles. But in the end, you still decide, what/where/how and when. In coding terms, you'll be coding the green blocks in the pyramid, and your artist (or yourself) are creating resources such as 3D models, world-maps, audio libraries, textures, and so on. The stuff that makes a game.

Ok. Obviously the way how this pyramid is constructed, is pretty vital if your plan is to make an easy-going, robust engine. Wrong relations between modules will reduce reusability. Ambiguous module content will make the engine hard to comprehend and master for other users. Making it too abstract won't accelerate end-users in making their game much. Focussing it too much on a very specific goal will filter out potential users as they can't use it for their own purposes. Yin and Yang balance is key my friends.

The Engine22 design is pretty much done. Probably I miss a couple of modules later on the ride, but this pyramid picture isn't far away from reality. Any specific why's here? Nah, not really. It's based on accumulated knowledge (& failure) from previous engine attempts and using other libraries - not necessarily game libraries. If I learned one major thing about design, is that ease & logic sense double matters. If I have to study some class or function longer than a minute, it sucks. Bottom line. Of course, it’s impossible to make each and every function "simple". Especially in a large, complex, multi-discipline case such as a game-engine. But at least we should try to keep things as simple as possible. Keep the names, relations and purpose of your modules/types/functions simple. And it's often better to provide just one or two "working" ways to accomplish X, instead of drowning the end-user with a billion different ways. Last but not least, tools, examples, documents and somebody to ask your questions will make a difference.

The first bricks
So, what should I be writing first about this engine? You could just as well ask which brick to lay down first for that pyramid. There is no single point to start. But I tried to start with the "fully isolated" modules first. With that I mean modules that don't require another. See that "Util" module? That was my first one; Basic types, Vector types, Matrices, and various (string/math/system) functions. Simply, but really useful. And boring. If you want me to write about engine stuff, you probably prefer awesome stuff like Physically Based Rendering or monsters fighting with each other. But the sad truth is that we'll have to start with the boring basic blocks first. There won't be any elite troops rappelling down from a cliff as long as we don't have:

If just one of those components is missing, the scene won't work or looks like shit. Realizing that, making a game can be demotivating. Having to make three billion other things first before you can even do relative simple things. But people often assume that all of this background stuff is "just there". But really, no. It takes a mountain of "boring" code -stuff you can't show in a slick screenshot- first... Exactly a good reason to pick an existing engine that did (most) of that stuff for you already.

As an engine-user, you don't have to know what happens behind the curtains. Sure, every extra bit of knowledge makes you a better person, but since women can drive cars without even the slightest idea of the motor-vehicle concept, so should you be able to make a game or 3D project without having to understand every detail. So in that perspective, maybe I shouldn't bother you too much with the deeper layers of Engine22, and focus more on the pretty things that can be produce with it. Only problem... I'm not that far yet! At least not on a level we reached with the previous Tower22 engine, which was capable of showing (seemingly) finished scenery. And in case you're that kind of guy who knows what's happening under the hood of his car, you may actually do want to know the "boring bits".

Modules, modules... what's a module?!
All right then. I already wasted a few pages on the introduction, a bit too late to dive into a specific engine module. So instead, let's explain what an Engine22 module itself is then. When I started coding, I had to decide how these "modules" would be made, programmatically. You see, there are many ways to put it all together. Some call it "modules", some call it "libraries". Some call it "API", some a "Framework", others "SDK" or "Toolkit". And what they exactly are in terms of programming, can also differ. We could simply attach a bunch of code files, use DLL's (Dynamic Link Libraries), or use "Components" in the Delphi environment. Now what's it gonna be boy?

Probably you already got it, but let's start with my (not "THE") definition of a Module. Here it goes: A collection of functions or types that can be used to achieve a specific goal. Do physics, do 3D rendering, do math, do scripts, whatever. A module doesn't do anything by itself, it has to be utilized by some other (higher) module or application. We try to setup modules as isolated, thus with as little dependencies on others as possible. Modules are programmed as separate projects, thus not as a bunch of cooperating files in a larger project. Ok? Fine.

Now in general, there are 2 ways to include a module; Statically or Dynamically. Static linking basically means all the projects get merged into one executable. Dynamic linking will keep the modules separate, so your executable has to link with them when it starts (or later if you prefer, eventually optionally).

Probably I get my ass kicked by some of the stated pro's and con's, but it should sum it up more or less. To me, the biggest advantage of using DLL's, is that we can use them in other Non-Delphi programs as well. Java, C, .NET, your choice. You can't statically include Delphi code into a C++ project (although... maybe with Embarcadero C++ Builder...). Yet I chose for a Statically linked approach. Why? Mainly because "Eazy doez it", and because I would lose a lot of OOP/Delphi features otherwise. There are work-arounds, but basically you can't share a TSomeClass. If you look into the OpenGL or Newton API for example, you'll notice this is merely a bunch of (C compatible) functions. Nothing wrong with that, but for a game with lots of "real-life-like" entities, I found objects & classes more intuitive. Besides, having Engine22 working in a C++/Java/Whatever environment is not an absolute must. Plenty of other *good* engines for these languages out there. And last but not least, Engine22 has an API module that still allows scripts and programs written in another language to use the engine. Ultimately, wrappers can be made of course.

Anyhow, Statically linked modules it is. As for Delphi, each Module is made as a BPL package project. You include E22 in your program by simply adding the "/source/" folder of each required Module in your project "Search folders". And then include the units you'll need, usually starting with the "Core" unit that acts as a main entry point to all other modules and stuff. Well, don't worry, examples will follow.

Technically, I'm not really utilizing the capabilities of a Delphi BPL package. Honestly, being new in Delphi XE as well, I wouldn't even know how to properly do that! (so Delphi guru's, help is appreciated here!!). I guess there are nicer ways to just activate a package, instead of having to manually include all /source/ folders. But that's a detail. Once it has been set-up, you're ready to play. Oh, and since we're statically linking everything, notice that A: the Source will be open, and B: there won't be any additional library (DLL) files you'll have to deploy with your executable.

Well... almost none. Because a few 3rd party modules are actually linked dynamically. I'm talking about OpenGL (3D graphics), Newton (physics), LUA (scripting), and FMOD (sound, optional). Note that FMOD isn't free. Or at least, you can download & use it for free, but you'll need a license when using your program for commercial purposes. Now I like FMOD, but I should be providing support for a free alternative -OpenAL (not to be confused with OpenGL). As for OpenGL for graphics, usually this is installed & updated automatically. But since E22 will be using modern techniques such as tessellation and bindless graphics, you'll be needing a relative modern video-card as well. One that supports OpenGL 4.3 at least.

When can we download Engine22 & start doing stuff? Well, we got webspace , and I'm writing beginner tutorials as we speak. It's just the engine as a whole is far from finished, so I don't want to upload a very buggy version to begin with. Then again it shouldn't take long anymore, just as long you don't expect a full working right from the start. If you do want a solid version, it's probably better to wait a bit longer :)

Certainly not back at the old Tower22 level, but at least there is progress. And quite rapid progress I should say. What you're seeing here? A shot of the Map Editor, Parallax corrected Cubemap reflections, a bit tessellation, HDR, SSDO (improved SSAO), Tiled Deferred Rendering, and some first bits on Image Based Lighting (IBL), using the Cook Torrance lighting BRDF and probes to sample reflections and irradiance. Now that's a mouthful. And Yes, this editor will become available too, but likely a bit later than the source code.