Xe03 Spectrum

XEO3 Lives!

2007-12-16T20:57:00.000Z

Well I will forgive you for thinking that this project has gone on indefinite hiatus, but it has not been forgotten, when I have had the time (we have had a daughter, now 6 months old, so my "free" time has been limited), I have been moving the project to my new assembler, RKASM.

I am pleased to report that it has all been moved over, and while I would like to do a bit of optimisation (plenty of scope for that), I have it all compiling and back in ship shape.

I've actually had it mostly done for a while now, but I could not get it running on the emulator... turns out I had just one or two bits wrong in the SNA file format

No system variables in the output file... caused interrupts to go haywire
Wrong page value written out for port $7ffd
Wrong interrupt mode value....

Once I had sorted these out it worked like a charm and then I just had a couple of crashes to sort out, just little bugs in the assembler... but these are all sorted.

Now I want to add a few items to the assembler to help with packing all the code in (there is a lot of wasted space at the moment), so I am going to add in sections, like I would use with a linker.. but I am going to omit the linker and just build it into the assembler so it will reorder the sections before saving out.

At that point, I'll fix the 128K code (currently broken) and sort out how I will work with the level data (looking at having everything loaded and not have any loading going on, using a single 128K snapshot).

Once I have that in place I will then be in a position to add content and work on the first level!

I think I'll even put the assembler up as a download for anyone else sad enough wants to use... once I have speeded it all up, shamefully it is taking a full 3s to assemble the code (not very big)... but I know that this is mostly going on tokenising the input stream, as that is painfully slow (and written far too quickly), the good news is that this is easily fixed... I just need to find the time to do so... probably over Christmas sometime.

RKASM!

2007-09-12T16:50:00.000+01:00

Well Mike has been complaining that my posts are very sporadic (he's a fine one to talk!). So I thought I would update you all on progress since SigGraph...

Well while I was away I implemented the sprite movement and bullets (including the collision detection), so I now have a pretty full game system in there now...

But I was getting very annoyed with the assembler that I have been using, mainly because I wanted to have a linker and a lot more power for putting everything together. I decided to take a leaf out of Mikes book and write the tools that I need.

So I've been writing a Z80 assembler for the Spectrum that does exactly what I wanted it to do, it is called RKASM (Russell Kay assembler) which is not very original.

My main goals just now are to get something that can assemble the code that I have already, for that I need
* basic instructions
* ORG, EQU, ALIGN
* SNA output.
* macros
* structs

I have the top 3 done already and I'm working on the last 2.

After that I'm going to add sections and linking semantics into the assembler, so that I can then organise things to my satisfaction within the game.

Live from San Diego!

2007-08-07T23:42:00.000+01:00

Well Mike and I are both in San Diego at SigGraph getting our brains filled up with knowledge once again. There are lots of amazing papers, tricks and techniques that are being discussed here, unfortunately not too many that are applicable to the spectrum. It is amazing to think how things have moved on in a relatively short period of time, most of the things being discussed were impossible (at realtime rates at least) even a few years ago, never mind 25 years ago.

Anyway I have had a chance on the plane over and between some sessions to add a functional section of the path moving code, so I can now have (logically at least, not hooked it up to the render code yet, though it is trivial) sprites moving around the screen, following paths that are specified in the level data.

I have followed Mikes layout as much as possible so that we can share a lot of the path data... It will not always be possible because of the resolution differences but I will where possible.

Anyway I'll post more details in the near future.

Lists

2007-07-16T20:41:00.000+01:00

So the sprite cache is in, I thought I should tell you all how it works, for the record.

The sprite cache is used to maintain a least recently used list of all the entries so that we can reuse an entry that will have as little impact as possible, the whole point of the sprite cache is to minimise the work done when drawing a sprite.

The main data structure is a doubly linked list that is setup like a queue where we know the top item and the end item. So each time a sprite is rendered it is moved to the top of the queue, then when we need a sprite entry we use the end of the queue.

Now normally a doubly linked list needs to be updated in an operation like moving to the top of the list requires some coniditionals (for updating the head and tail of the list). So in C# (I like to prototype the ideas first and these days C# is the language of choice).

so in this snippet m_cachePrev and m_cacheNext are arrays of ints (pointing at the previous and next entries in the list, First is the first entry in the list and End is the last entry in the list.


        public void MoveToTop( int _index )
        {
            // unhook previous entry
            int prev = m_cachePrev[ _index ];
            int next = m_cacheNext[_index];

            if (next != End)
                m_cachePrev[next] = prev;
            else
                Last = prev;

            if (prev != End)
                m_cacheNext[prev] = next;
            else
                First = next;

            m_cachePrev[_index] = End;
            m_cacheNext[_index] = First;
            m_cachePrev[First] = _index;
            First = _index;
        }

Now on the 6502 because the flags are set on load then Mike can get some benefits and speed ups, so this type of code is fine for Mike, but on the Z80 the flags are not set when an entry is loaded but only after specific operations. So I have been looking for a method of doing this that minimised the branching to make the Z80 code a bit nicer, well my first area of attack was on the First and End entries and the checks that go in there.

So why do we use End as an indicator that this is the start or end of the list? Well just convention really (normally End would be set to NULL) but if End was set to an entry within the array then First and Last could be set to those entries. So I have set End to be the last index of the array and allocate an extra entry (so if we have 128 entries in the cache then the 129th entry becomes the First and Last entries)

First == m_cacheNext[ End ];
Last == m_cachePrev[ End ];

then we can lose the conditionals because the actions would be the same. So the code becomes in C#


            // unhook previous entry
            int prev = m_cachePrev[_index];
            int next = m_cacheNext[_index];

            m_cachePrev[next] = prev;
            m_cacheNext[prev] = next;

            m_cachePrev[_index] = End;
            m_cacheNext[_index] = First;
            m_cachePrev[First] = _index;
            First = _index;

and then in Z80 this becomes


DSO_MoveToTop: 
             ld  h,high sprite_cache_entry_prev
  ld  b,h
  ld  l,c
  ld  e,(hl)       ; e = prev
  ld  (hl),SPRITE_CACHE_END   ; m_cachePrev[_index] = SPRITE_CACHE_END
  ld  h,high sprite_cache_entry_next
  ld  d,h
  ld  c,(hl)       ; c = next
  ld  a,e
  ld  (bc),a       ; m_cachePrev[next] = prev
  ld  a,c
  ld  (de),a       ; m_cacheNext[prev] = next
  ld  e,SPRITE_CACHE_END
  ld  a,(de)       ; a = First
  ld         (hl),a                ; m_cacheNext[_index] = First
  ld  c,a
  ld  a,l
  ld  (bc),a       ; m_cachePrev[First] = _index
  ld  (de),a       ; First = _index

If you spot any optimisations then let me know.

Progress!

2007-07-11T20:28:00.000+01:00

Well the sprites have been in for some time now and I've taken the time to reorganise the code and tidy things up and do some tests.

I can get 10 masked sprites on screen and still have time to render bullets and even some map sprites (these are cheaper to do as they do not need to be masked, but they do need to be ORed), so I feel fairly comfortable with where I am and I could do a fairly faithful representation of the Plus4 version.

The main difference is that 10 - 16x16 sprites makes the screen fairly busy, as the Plus4 has a wider effective screen than the Spectrum, so I might revise that down for aesthetic (rather than technical reasons).

So the next thing to do, are the path code for the enemies and it will start to feel like a game.

Now with added sprites!

2007-07-05T12:48:00.000+01:00

I now have a simple sprite routine up and running as outlined below, it all seems to be working pretty well and the sprite cache is working too which is a double plus.

Currently it is not using masks or anything like that so it all looks very dull, but all the logic is in place now.

I have to tidy up a few things, and I'll post the sprite cache linked list code in the near future as I have managed to make it branchless which improves the readability of it and I believe performance on the Z80 (as the flags are not set on load for Z80, so I have to introduce an instruction to check for zero).

Anyway, progress is being made and I hope to have something with several sprites flying around over the next few days.

And then I'll work on getting the path code working so that we can have formations of enemies.

I'm Back!!!!

2007-04-25T17:20:00.000+01:00

Well it has been a while... but I'm back on the Spectrum XeO3, so many things have intervened in the meantime between work and home, but I'm back looking at the Sprite routines for XeO3 and I'm looking to get them sorted out in the next couple of months (being realistic about the time I can devote to the project).

So major news to date is that I'm going to be a Dad again soon, so we have been preparing for that (and it gives me deadline for sorting out these pesky sprite routines, before changing nappies takes over).

Anyway Mike and I have been talking about the sprite cache system that he has been using on the Plus4 and I think it is the way to go for the Spectrum too, so I've been looking at his routines and seeing what would be the best way to translate them into Z80.

In Pseudo code the DrawSprite routine looks something like this

  Draw Sprite
  Calc sprite function to use (could be hard coded to Draw Sprite routine name).
  Calc address on screen
  if sprite_number+rotation not in cache
   [Add to cache]
    Get lowest entry on the stack
    Remove entry from the cache
    Add this sprite to the cache
    Rotate the sprite and put it into the cache.
  [Get address in cache]
   Move this cache entry to the top of the list
   retrieve sprite data address
  Push information onto sprite stack

This uses the sprite stack that I talked about earlier to do the actual rendering, but caches the rotated graphic for each sprite entry, this enables the use of lots of different sprite types without having to store all the possible rotations in memory at once (they are only rotated when required). Usually in the bad old days graphics would all be pre-rotated and restricted at what x coords they could be rendered at (Miner Willy springs to mind, his walk animation required it to move across the screen).

So I'm going to write the general sprite cache routine over the next week or so and then spend a few weeks debugging it properly, so that the game code can just issues a simple DrawSprite call and everything will just work...

It's been a while

2006-11-21T22:35:00.001Z

Well I thought I should write something as it's been a while since I last posted.

Currently I'm writing some tools to convert map data into the correct format for the scrolling routines and eliminate a lot of the mistakes I've been making because I've had to type the data in by hand. It's taken a while because well ... no excuses really I've just not had the time recently.

Anyway I've gotten it mostly written now C# is a wonderful thing (best thing MS have done in a long time) so XeO3Gen now exists to convert maps into the correct format. Now I just need to modify the code to use the new format (that's tomorrows job).

And for the trivia fans the name XeO3Gen is an homage to the tool I wrote for Lemmings 2 (l2gen) that converted all the graphics and style files for the target machines.

It's been a while

2006-11-21T22:35:00.000Z

I've always hated the taste of SPAM

2006-11-12T21:15:00.000Z

First time I've seen SPAM on a blog....

Anyway I've realised that I need to address the amount of time and memory that the scrolling graphics is going to take up, so I've been thinking how to change the routines as it stands (currently the scrolling is taking around half a frame of processing), now I know that I can improve that by storing the graphic pre-rotated but that means that each set of 4 tiles (they need to be grouped together0 will take 2K for all 16 rotations, which means that on a 48K Spectrum I would get very uninspiring graphics.

Well I think I have a new method but it will take a bit of experimentation.. so I'll keep you up to date with the progress.

Xe03: Sprites

2006-11-09T21:24:00.000Z

Well I've not fallen off the face of the planet, Mike and I have been busy beavering away and we've both been looking at the issue of sprites, I've been unsure on the best way to do them on the Spectrum and wondering how he has acheived so much on the Plus/4. Anyway see Mikes blog for some of the fruits of those discussions about different techniques.

So now that I have the scrolling working and rendering I want to draw as many sprites as possible (Mike manages 10 masked sprites on the Plus/4) onto the main play area. On the 48K (which is still my target, as much as Mike keeps saying that I should look at 128K). So to achieve this I have to ensure that I draw the sprites before the raster displays the area that I am rendering to. Therefore I need to draw the sprites in a sorted order (from the top of the screen to the bottom) to stay ahead of the raster. Since the scrolling takes from the top of the screen (where I sync to) to the bottom of the screen (well a few scan lines over) I have the whole of the lower border, VBlank, upper border and then the screen area (staying ahead of the raster). Which is a fair amount of time...

So I have to get a really fast sprite routine, and I invite you all to give suggestions on the best way. Following some discussions on the World of Spectrum (www.worldofspectrum.org/forums) I'm using the zig-zag pattern of sprites i.e. for a sprite that is of the format

ABC
DEF
GHI

then the bytes would be arranged normally as

A0 B0 C0
A1 B1 C1
A2 B2 C2
...
G7 H7 I7

In Zig Zag they are arranged as

A0 B0 C0
C1 B1 A1
A2 B2 C2
C3 B3 A3
...
I7 H7 G7

In this way it minimises the number of increments and decrements that a sprite routine has to do (also means that the screen address does not need to be reloaded at all).

So the basic structure of a sprite routine is

// byte 0
    get source
    logical operation with dest
    store in dest
    next source
    inc dest
// byte 1
    get source
    logical operation with dest
    store in dest
    next source
    inc dest
// byte 2
    get source
    logical operation with dest
    store in dest
    next source
    move dest down one line
// byte 3
    get source
    logical operation with dest
    store in dest
    next source
    dec dest
// byte 4
    get source
    logical operation with dest
    store in dest
    next source
    dec dest
// byte 5
    get source
    logical operation with dest
    store in dest
    next source
    move dest down one line

So going with my old favourite, I'm going to use the stack pointer to point at the source data and then store my Mask and Data together so that a single pop will read both of them and my inner core will become

     pop   de
     ld    a,(hl)
     and   e
     or    d
     ld    (hl),a
     inc    l

Using the stack means that we fetch the data and increment to the next byte all at once, the logical operation is fairly simple and the step to the next screen byte is a simple increment all nice and easy!!

Now I've done some timings and I can get around 14 (24x16) of these in our time frame which is pretty good.

The observant among you will have noticed that I'm not doing any rotation in this routine, that's because I'm going to implement a sprite cache like Mikes and store all the sprites rotated in the cache (remaking them when I need to), which means I don't need to do that in here (remember staying ahead of the raster).

Lastly I need to feed these routine as quickly as possible so I am using a small routine that reads a table and calls the correct draw routine as quickly as possible

DrawSprites:    di
                ld   (DS_sp_restore+1),sp

                ld   de,6
                ld   hl,SpriteStack
DS_lp:          ld   sp,hl
                exx
                pop  hl   ; routine address
                ld   a,h
                and  a
                jr  z,DS_exit

                ld  (DS_jump_address+1),hl
                pop  de   ; screen address
                pop  hl   ; graphics address
                ld   sp,hl
                ex   de,hl
DS_jump_address jp   #ffff
DS_sp_lp_back:  exx
                add  hl,de
                jr   DS_lp

DS_exit:
DS_sp_restore:  ld   sp,#ffff
                ei
                ret

So if you can spot a quicker way to do things let me know, because I want to make the Spectrum version better than the C64 or Plus4... ;)

Scrolling Working!

2006-11-03T09:01:00.000Z

Well I got the scrolling working yesterday, so I thought I would put a video up for you all to see, still programmer art so it does not look very nice, and I have a particularly bad sprite routine in there just now (I know I can optimise the heck out of that one), I've been mainly worried about the scrolling up to now.

It is still taking a bit too much time just now, but I'm going to get the sprite routines in and start adding some gameplay in there before I go back and make more decisions on the scrolling.

Currently I have a cache of 16 tile pairs per screen that I am scrolling (as I'm trying to avoid storing prerotations) and 4 tile pairs per screen line, but I know I can expand that to 8 tile pairs per screen line, and I can move to pre-rotated tiles if need be (to lift the 16 tile pairs restriction). So I am fairly comfortable with this.

Also if I need them I can overlay sprites on the background as well to make it look nicer if need be, though my aim is to have as many sprites moving as possible to keep the action frenetic.

Anyway it's just a little taster to show progress... and remember I am no artist....

Scrolling routine

2006-10-26T19:50:00.000+01:00

So I thought that I would discuss the scrolling routine to let everyone see what I'm doing, on the Spectrum there is no hardware so we have to do everything with the processor so I have to re-render the whole scrolling area as it is constantly moving, so in pseudo code the routine is.

                 Initialise
outer_loop:      Read jump Address, graphic address and screen address from table
                 if jump address high byte is 0 then jump to exit
inner_loop:      Pop graphics into registers
                 jmp Jump address
return_from_jump:advance to the next line of the screen
                 jump to outer_loop 

exit:            restore
                 return.

jump_address:    push registers to screen
                 jump to return_from_jump

As I have said before the fastest way on the Z80 to access memory is to use the stack, it takes 11T states to pop 16 bits from memory, whereas it takes 20T states to read 16 bits normally with a "ld hl,(NN)" instruction. The problem with using the stack pointer is that it means that we cannot do anything that would use the stack (i.e. call sub-routines or allow any interrupts), so initialisation looks like this

Scroll_Render:  ld  de,SR_renderData
                di     
                ld  (SR_sp_restore+1),sp     
                ld  (SR_outer_loop+1),de

So this little bit of code disables the interrupts and then saves the sp register into some self modifiying code to restore the current stack pointer value, then saves the address of the table (SR_renderData) into another bit of self modifying code. The Z80 has very limited instructions for setting the value of sp only

  ld sp,NN
  ld sp,(NN)
  ld sp,hl
  ld sp,ix
  ld sp,iy

so to set sp to a definitive value I use the ld sp,NN instruction and then set the value of NN by writing to that address (the address of the opcode +1).

The outer loop then has to read the values from the table

SR_outer_loop:  ld   sp,#ffff     
                pop  hl     
                ld   a,h     
                and  a     
                jr   z,SR_exit
                ld   (SR_prepare_jump+1),hl     
                pop  ix     
                pop  iy     
                ld   (SR_outer_loop+1),sp

So the first ld sp,NN is initially modified with the the start address of the table, the values are then popped of the stack (advancing sp at the same time) and then the last instruction stores the new value of sp back in the instruction that loads sp for the next time around the loop. Oh and a jump instruction (coming up in the next section) is then modified so that the jump goes to the correct place.

SR_inner_loop:  ld   sp,ix     
                pop  hl     
                pop  de     
                pop  bc     
                pop  af     
                ld   sp,iy
SR_prepare_jump:jp   #ffff
SR_jump_return: ld   de,8     
                add  ix,de     
                inc  iyh     
                ld   a,iyh     
                and  7     
                jr   nz,SR_inner_loop     
                jr   SR_outer_loop

In this section we prepare the registers with the graphics for this line of the screen, the IX register holds the base of the graphics which is then used to set sp, the graphics are popped into the register and then sp is set to the screen address (IY) then we jump to the correct routine (self modified from earlier), the return address from the routine is just below which advances the graphic address and then the screen address is incremented to the next line down, then we keep going until the end of the table.


SR_exit:
SR_sp_restore:  ld  sp,#ffff     
                ei     
                ret

This section just restores sp back to it's original address, renables the interrupts and then returns back to the calling routine (in this case the main loop).

Now the routines that are called for rendering each line, will actually be constructed from the map section that we want to render so these will change all the time (and I'll be writing some code later to create them), but they look like this

SR_line_00:   push hl
    push de
    push bc
    push af
    push hl
    push de
    push bc
    push af
    push hl
    push bc
    push de
    push af
    push hl
    push bc
SR_line_00ret:                  jp      SR_jump_return

This code is relatively simple and since each opcode is only 1 byte it will be relatively simple to construct, the jp address always returns to the same place for each bit of code so that is even easier.

Now those who are still awake and have reached this section will have noticed that I have avoided using the exchange register set in this routine completely, this is because currently I am only using 4 different graphic tiles per line for the map but if I use the exchange registers then I could get a total of 8 different tiles on each line. But I have to trade that off against the amount of time it will take to setup i.e. 8 pops and not 4 as it is just now (and also complicating the line rendering code as it will have to swap the register sets as well as it push's), this will slow down rendering. I do have an option though of moving the line setup code into the rendering as well, this would be relatively simple and mean that only the registers that are used would be popped (so if only one is used then only that one is popped) this would give a lot more flexibility.

Anyway thats all for just now, back to the assembler...

All change

2006-10-26T13:31:00.000+01:00

Mike has been educating me on the wonders of Cascading Styling Sheets so it's all change today on the blog.

Last night I optimised the rendering code quite a bit and managed to shave off a lot of time (around 40 scan lines!!!) it was all down to reorganising things and using tables rather than calculating.

The picture attached shows how much time I've saved in the timing bars (cyan is the render time for the scrolling area, red is the sprite rendering time) so you can see the change from the last post.

So I have now moved some of the calulcation to tables and I've made better use of the available registers. Tonight I'll post the code for everyone to look at (and criticise :) )

Pictures!!!

2006-10-23T20:55:00.000+01:00

Mike has been pestering me to put up an in-progress picture, and I have to emphasise that this is very much in-progress (including programmer art).

Basically this shows the scrolling area being redrawn every frame with a sprite being rendered on top (actually it is currently 4 sprites in the same position).

The border colours are showing how long is being taken for the 2 routines (Cyan is the scroll render and red is the sprite drawing). So the scroll render is taking around 2/3rds of a full frame and we are only rendering the top 2/3rds of the screen (128 pixels).

I've been talking tactics with Mike and I've had to rethink some things over the weekend as the Z80 was just not fast enough for what I was thinking... it really take some time to readjust to how little can really be done by these processors, we really are spoiled by current machines!

I still have to write the processing part of the scrolling which will create the code to render the screen and rotate the tiles of the actual screen, but we have already decided that the game will run in 2 frames (i.e. it will update at 25fps, the same as the Plus/4 and the C64) so I have plenty of time.

Mike also pointed out that the 128K Spectrum would allow me to double buffer the screen (which would mean I would not need to use the floating bus technique), currently I'm a purist aiming at the 48K Spectrum but I will be doing a 128K version as well so I'll keep that knowledge in my back pocket.

I need to get back to this processing stuff now so that I can get it all moving...

Old and New

2006-10-18T21:12:00.000+01:00

So I've spent a while now implementing scrolling on the Spectrum and its the first Z80 code of any size for a long time and I'd forgotten how processors have moved on over the years. I've been frustrated by little things that you take forgranted on x86, Mips etc.

1. No instruction to add a constant value to a register, only register to register.
2. Rotate instructions only rotate by one at a time... oh to be able to rotate by a variable amount.
3. Lots of registers but a lack of addressing modes, a lot of instructions only allow register to register interactions and that means you need to shuffle them around all the time (and a lack of shuffling instructions (at least for 16bit regs)).

Some things are good though, the ability to use the flags registers as return states (mainly because a lot of instructions do not set them), this makes things much easier in many ways.

More modern processors make it much easier and what is natural these days is far too slow for the old processors, I need to think much more about minimising the amount of work that needs to be done by the Z80, a readjustment in my thinking is needed...

Waiting for the right moment!

2006-10-14T21:03:00.000+01:00

The Spectrum is an interesting beast because it basically has no hardware and you need to do everything in software. When rendering the best way to do things is to render behind the raster that is sending everything to the TV screen.

The only synchronisation that is available is with the vertical sync of the raster, but this leaves a lot of time (in terms of machine cycles) before the start of the screen, where we want to start rendering to allow us to draw just behind the raster, if we render before hand then we will see flicker, just behind and the display will be rock solid (which is what we are aiming for).

Fortunately on the Spectrum there is a little known way that this can be done, once again a technique which was used on Cobra (a technical masterpiece), it uses the floating bus on the Spectrum (see http://www.ramsoft.bbk.org/floatingbus.html for an explanation). The code is relatively simple


SyncToScreen: ld bc,#3f28
STS_lp:       ld a,c 
              in a,(#ff) 
              cp b 
              jr nc,STS_lp 
              ret

This routine basically waits until an attribute is being output from the ULA of the Spectrum, which if it is started during the border region of the display, then it will wait for the first line of the screen... exactly what we need!

So now our armoury of techniques is filling up I really need to get on and write some real code...

First Post

2006-10-11T22:00:00.000+01:00

Well here we are, first post on a journey back into time.

It's been around 20 years since I last touched a real Spectrum to program it, but here I am back again!!!

Mike has given me this chance to put together a Spectrum version of Xenon TriOxide, so I've been re-acquainting myself with an old friend.

The game is a scrolling shoot em up, and the best scrolling method on the Spectrum was used in Cobra, where the screen is redrawn every frame by "push"ing the map onto the screen.

The technique is fairly simple the stack pointer is set to a screen address and then code is executed which draws the screen, this code is created procedurally from the game map, it is a simple tower of push instructions, in psuedo code it is reasonably simple

disable interrupts
store SP into memory location (of the ld sp,NNNN instruction below)
set SP to screen address (right most column of screen)
initialises registers with the graphics
Execute the constructed code
Loop for all the screen lines that are to be rendered
ld sp,NNNN (this instruction will now contain the stored SP register)
enable interrupts

This is the fastest way to redraw the screen.

Tomorrow I'll talk about how to synchronise the screen so that we don't get any flicker.