tag:blogger.com,1999:blog-67677072024-03-06T02:41:17.308+10:30Rafe's BlogI'm pretty hopeless at keeping this thing up-to-date...Rafehttp://www.blogger.com/profile/10169325688704651715noreply@blogger.comBlogger106125tag:blogger.com,1999:blog-6767707.post-56542733261104606742022-02-01T21:51:00.000+10:302022-02-01T21:51:13.356+10:30Kate Clanchy & the Social Justice Bastards<p> <a href="https://www.youtube.com/watch?v=wq3VZaVRj6o">Kate Clanchy: "My life's work has been taken away" - YouTube</a></p><p>These hyper-woke people can't <i>all</i> be mentally ill. There seems to be an element of deliberate evil in all this.</p>Rafehttp://www.blogger.com/profile/10169325688704651715noreply@blogger.com0tag:blogger.com,1999:blog-6767707.post-17255144242035855852021-03-08T09:09:00.002+10:302021-03-08T09:09:19.467+10:30Social "Justice" at Smith College<p>Unconscionable. I challenge any of my SJW aligned acquaintances to defend any of this.</p><p>https://www.youtube.com/watch?v=IZZ1VC44k9k</p>Rafehttp://www.blogger.com/profile/10169325688704651715noreply@blogger.com0tag:blogger.com,1999:blog-6767707.post-57599180631785051482019-04-19T09:00:00.001+09:302019-04-19T09:00:36.319+09:30More on drawing sprites on the ZX SpectrumA recent discussion on the <a href="https://www.facebook.com/groups/z80asm/?ref=group_header">Z80 Assembly Programming on the ZX Spectrum</a> Facebook group prompted me to look into the differing costs of various ways of drawing sprite images on the ZX Spectrum.<br />
<br />
The discussion revolved around handling the Speccy's slightly odd display file layout. To recap, the bitmap display consists of 24 rows of 32 columns of cells, each cell of which has 8 lines of 8 pixels (one byte per line). The address for a particular cell line is <i>010RRLLL RRRCCCCC </i>where <i>RRRRR </i>is the row number (0..23), <i>CCCCC </i>is the column number (0..31), and <i>LLL </i>is the line number (0..7), all in binary. Note that the display addresses are broken into three sections under this scheme: rows 0..7, rows 8..15, and rows 16..23.<br />
<br />
(There is a separate attribute overlay giving the foreground and background colour for each <i>cell</i>, but I'm not going to discuss that here.)<br />
<br />
The advantage of this layout is that if the display address is in a Z80 register pair, <i>HL </i>say, one can move from one cell line to the next simply by incrementing the upper half of the register pair: <i>inc H</i>.<br />
<br />
There are three main ways of drawing a sprite image on to the display. For the sake of argument, I will assume a 16x16 pixel image (more or less standard for most games on the Speccy). I am not going to consider clipping at the edges of the display. I am going to assume that we will draw images to the display using the <i>xor</i> method (very common: redrawing an image in the same place has the effect of removing it from the display). I will further assume that, for speed, each image comes in 8 pre-shifted forms:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">xxxxxxxx </span><span style="font-family: "courier new" , "courier" , monospace;">xxxxxxxx ........ </span><span style="font-family: inherit;">- x 16 lines for 0px offset</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">.xxxxxxx x</span><span style="font-family: "courier new" , "courier" , monospace;">xxxxxxx x....... </span><span style="font-family: inherit;">- x 16 lines for 1px offset</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">..xxxxxx x</span><span style="font-family: "courier new" , "courier" , monospace;">xxxxxxx xx...... </span><span style="font-family: inherit;">- x 16 lines for 2px offset</span><br />
<span style="font-family: inherit;">.</span><br />
<span style="font-family: inherit;">.</span><br />
<span style="font-family: inherit;">.</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">.......x xxxxxxxx xxxxxxx. </span><span style="font-family: inherit;">- x 16 lines for 7px offset</span><br />
<br />
(1) Break the image into cell sized chunks and draw each cell as a separate operation. This means that a 16x16 image at an arbitrary pixel position on the display will be drawn over a 3x3 cell region (i.e., 24 lines of drawing for 50% over-draw). I have discussed this at length here before.<br />
<br />
(2) Draw only the lines in the image (i.e., no over-draw), dealing with the line/row/third transition with in-line code.<br />
<br />
(3) Draw only the lines in the image, but use a separate table of display line addresses (i.e., pre-computing the line/row/third transitions ahead of time).<br />
<br />
So how do these schemes work out? (I'm using Simon Brattel's excellent <a href="http://desdes.com/products/oldfiles/index.htm">Zeus </a>assembler syntax here, in case anyone is wondering.)<br />
<br />
<b>Dealing with the line/row/third transitions with in-line code.</b><br />
<b><br /></b>
The best approach I could come up with here was this:<br />
- unroll the drawing code for 8 lines at a time;<br />
- handle the row/third adjustment after the 8th line;<br />
- jump into the correct spot at the start (e.g., if the top line of our image starts at line 3 of our cell, then we would start drawing at the "line 3" entry point).<br />
<br />
Here's my code to do that:<br />
<br />
<pre>; DE points to the image data: L-to-R, T-to-B, three bytes per line.
; C is height of sprite in lines.
; HL is address of display byte for the top-left byte of the sprite image.
Xor24px_Adj proc
; Jump into the correct offset for row adjustment.
; Mean cost is about 18 + 3 * (4 + 7) + 1 * (4 + 12)
; = 67 t-states.
ld A, H
and 000111
xor 000111
jr z, _L7
dec A: jr Z, _L6
dec A: jr Z, _L5
dec A: jr Z, _L4
dec A: jr Z, _L3
dec A: jr Z, _L2
dec A: jr Z, _L1
; Unrolled drawing loop.
_XorLineAdj macro()
; Cost per 3 byte line is 3 * (21 + 6) + 6 * 4 + 5
; = 112 t-states.
ld A, (DE): xor (HL): ld (HL), A: inc DE: inc L
ld A, (DE): xor (HL): ld (HL), A: inc DE: inc L
ld A, (DE): xor (HL): ld (HL), A: inc DE
dec L: dec L: inc H
dec C: ret z
endm
_L0 _XorLineAdj()
_L1 _XorLineAdj()
_L2 _XorLineAdj()
_L3 _XorLineAdj()
_L4 _XorLineAdj()
_L5 _XorLineAdj()
_L6 _XorLineAdj()
_L7 _XorLineAdj()
; Adjustment for next row.
; Mean cost per line is (2 * (4 + 7 + 4) + 7 + 12) / 8
; = 6 t-states.
ld A, L: add 32: ld L, A: jr c, _L0
lf A, H: sub 8: ld H, A: jr _L0
; Overall cost after setup overhead is
; 112 + 6 = 118 t-states per line.
endp
</pre>
<br />
So, for 16x16px pre-shifted sprite images, we expect to take around 67 + 16 * 118 = 1,955 t-states per image with this approach (there are other overheads, of course, but let's take this as our baseline).<br />
<br />
Note: this runs to about 189 bytes of machine code, so it is quite compact.<br />
<br />
<b>Dealing with the line/row/third transitions with a pre-computed table of display line addresses.</b><br />
<b><br /></b>
The best approach I could come up with here is to pre-compute the starting address of each display line:<br />
<br />
<pre> dw $4000, $4100, $4200, ..., $4700
dw $4020, $4120, $4220, ..., $4720
...
dw $40e0, $41e0, $42e0, ..., $47e0
dw $4800, $4900, $4a00, ..., $4f00
dw $4820, $4920, $4a20, ..., $4f20
...
</pre>
<br />
then<br />
- point the stack pointer <i>SP</i> at the correct entry for the starting display line of the sprite image;<br />
- put the image column number in a different register, <i>B</i>;<br />
- pop each display line address off the stack (this is the fastest way of fetching 16-byte data on the Z80);<br />
- drawing code for the <i>entire</i> image is unrolled! If you want to draw a smaller image, you jump into the correct entry point for that.<br />
<br />
Here's my code to do that:<br />
<br />
<pre>; DE points to the image data: L-to-R, T-to-B, three bytes per line.
; SP points to the table of display line start addresses offset for
; the first line of the sprite image.
; B is the leftmost column number of the sprite image.
; User calls into the correct offset (e.g., Xor24px_Tbl_16 for a 16 line
; image).
_XorLineTbl macro()
; Cost per 3 byte line is 22 + 3 * (21 + 6)
; = 93 t-states.
pop HL: ld A, L: or B: ld L, A
ld A, (DE): xor (HL): ld (HL), A: inc DE: inc L
ld A, (DE): xor (HL): ld (HL), A: inc DE: inc L
ld A, (DE): xor (HL): ld (HL), A: inc DE
endm
Xor24px_Tbl_16 _XorLineTbl()
Xor24px_Tbl_15 _XorLineTbl()
Xor24px_Tbl_14 _XorLineTbl()
; ...
Xor24px_Tbl_3 _XorLineTbl()
Xor24px_Tbl_2 _XorLineTbl()
Xor24px_Tbl_1 _XorLineTbl()
; Exit code goes here (e.g., restore SP and ret).
</pre>
<br />
So, for 16x16px pre-shifted sprite images, we expect to take around 16 * 93 = 1,488 t-states per image. Again, there is a bit more overhead to take into account here, but I'll call this the base-line figure.<br />
<br />
Note that this scheme uses a lot more code: around 288 bytes of unrolled drawing code for a 16x16px image along with a 384 byte address table. For my money, that is a good trade-off for the extra speed.<br />
<br />
<b>Comparison.</b><br />
<br />
Rounding the numbers up, fixing things at the line-8 transition costs you 2,000 t-states per sprite image whereas using the stack-pointer-into-a-lookup-table scheme costs you only 1,500 t-states per sprite image.<br />
<br />
What does this mean in practice? Well, the usual approach to obtaining flicker-free graphics involves waiting for the screen-refresh interrupt, then redrawing everything before the CRT scan line catches up with you. On the ZX Spectrum the upper screen border (before the display bitmap is drawn) takes around 14,000 t-states. <br />
<br />
For the fix-in-code approach, this is enough time to erase and redraw 14,000 / 2,000 / 2 = 3.5 16x16px sprites before you risk being caught be the screen refresh.<br />
<br />
For the use-a-table approach, this is enough time to erase and redraw 14,000 / 1,500 / 2 = 4.6 16x16px sprites before you risk being caught by the screen refresh.<br />
<br />
<b>Conclusion.</b><br />
<b><br /></b>
For my money, I'd go for the table-based approach.<br />
<br />Rafehttp://www.blogger.com/profile/10169325688704651715noreply@blogger.com0tag:blogger.com,1999:blog-6767707.post-48929117664034794312018-07-18T15:05:00.000+09:302019-09-18T11:44:51.080+09:30High speed full-screen drawing on the ZX SpectrumThis is another retro-programming idea that occurred to me, harking back to my old ZX Spectrum days. If you're not au fait with the speccy's screen layout, I refer you to this posting of mine about <a href="http://ralphbecket.blogspot.com/2015/09/faster-sprites-on-zx-spectrum.html">rendering sprites</a>.<br />
<br />
The problem is that the old Z80 isn't fast enough to update all of the video memory before the CRT beam catches up with you. The screen has a 64 pixel upper border (requiring no video memory), followed by a 192 pixel high by 256 pixel wide bitmap, followed by a lower border (also requiring no video memory). The bitmap is monochrome, with each consecutive set of 8 pixels being represented in a single byte. There is a separate 24 row by 32 column attribute map specifying foreground and background colours for each 8x8 pixel cell.<br />
<br />
The only interrupt you get on the speccy occurs once every 50th of a second when the CRT beam starts from the top of the upper border (so this is pretty much when you have to start drawing). It takes 224 T-states (i.e., clock ticks) for the CRT beam to scan one pixel line. Since it takes around 16 Ts to move a single byte (give or take), the Z80 can update at most half of the 32 bytes on each row of pixels before the CRT beam catches up (which would appear as "tearing" on the screen).<br />
<br />
[Note: some games did manage a full 50 FPS refresh rate, but only by severely restricting the area of the screen that was updated.]<br />
<br />
However, let's say you only wanted to update about a third of the bitmap at 50 FPS, but didn't want to be restricted to which third. The remainder would be painted in the background colour, which is easily arranged by setting the corresponding cells in the attribute map to have the same foreground and background colour (i.e., the set pixels and the clear pixels in such an 8x8 cell will have the same colour).<br />
<br />
What occurs to me is this. For a "blank" cell, we just need to set the cell attribute. For a "bitmap" cell, we need to set the cell attribute <i>and</i> set the corresponding bitmap bytes in the video memory. Assuming that we have the address of the "current" attribute entry in DE and the bitmap entry in HL (and the magic number 1 - $0700 in BC), then we can do the following:<br />
<br />
DrawSomeBlankCell:<br />
ld A, SomeAttr<br />
ld (DE), A<br />
inc E<br />
inc L<br />
ret<br />
<br />
(for 32 Ts per blank cell)<br />
<br />
DrawSomeBitmapCell:<br />
ld A, SomeAttr<br />
ld (DE), A<br />
inc E<br />
ld (HL), SomeBitmapByte0 : inc H<br />
ld (HL), SomeBitmapByte1 : inc H<br />
ld (HL), SomeBitmapByte2 : inc H<br />
ld (HL), SomeBitmapByte3 : inc H<br />
ld (HL), SomeBitmapByte4 : inc H<br />
ld (HL), SomeBitmapByte5 : inc H<br />
ld (HL), SomeBitmapByte6 : inc H<br />
ld (HL), SomeBitmapByte7 : add HL, BC<br />
ret<br />
<br />
(for 146 Ts per bitmap cell).<br />
<br />
The "draw list" would simply be a consecutive list of pointers to these drawing functions:<br />
<br />
DrawList:<br />
dw Draw0, Draw1, Draw2, ..., Draw31, [fix up]<br />
dw Draw32, Draw33, Draw34, ..., Draw 63, [fix up]<br />
....<br />
dw ..., Draw767, [finish up]<br />
<br />
We would redraw the entire screen by simply pointing DE at the display attributes, HL at the display bitmap, SP at DrawList and executing a 'ret'. (The [fix up] calls would simply deal with the odd end-of-line addressing fixes that are needed when handling the ZX Spectrum display.)<br />
<br />
How much could we redraw, assuming an even distribution of action across the screen? A simple calculation shows we can have a good 11 bitmaps on each line and still update the entire display before the CRT beam catches up. That is just slightly in excess of 33% of the entire display!<br />
<br />
(I'm currently working on a full-screen Pac Man demo. to illustrate this idea. Time is scarce, so don't hold your breath.)<br />
<br />
A very belated update: you can find the working source code at <a href="https://github.com/ralphbecket/Z80/tree/master/MegaPacMan">https://github.com/ralphbecket/Z80/tree/master/MegaPacMan</a> showing a full-screen 50 FPS scrolling maze with four ghosts running around at random. It could support many more and/or much larger sprites, but I'm happy to call it a day here!Rafehttp://www.blogger.com/profile/10169325688704651715noreply@blogger.com9tag:blogger.com,1999:blog-6767707.post-25615663030907712017-11-17T22:52:00.000+10:302017-11-17T22:52:11.696+10:30Retro-programming: an High-Level Byte-Code for 8-bit CPUsThis idea has been percolating around in my hind-brain for a while now, so I thought I'd set it down on paper.<br />
<br />
<h3>
The Problem</h3>
Imagine you are back in the 1980s, programming one of the wonderfully primitive home micro-computers that has just arrived on the scene. Say, a ZX Spectrum, sporting a mere 48 Kbytes RAM and a 3.5 MHz Z80 CPU (alternatively, a comparable 1 MHz 6502-based Commodore 64). These CPUs are aggressively 8-bit in nature -- yes, the Z80 has some 16-bit support, but it isn't great. Worse, these CPUs are non-orthogonal: every register seems to have its own special rules. What would have been nice was some kind of easy-to-implement byte-code interpreter which would be comfortable to program in and for which it would be easy to write a compiler in BASIC (all home micro-computers booted directly to BASIC in those days). This virtual machine would, of course, run more slowly than hand-crafted assembly language, but many times faster than the interpreted BASICs of the day. Having a standardised byte-code would mean you could run the same program on any machine on which you had an implementation of the byte-code interpreter.<br />
<br />
[Note: these thoughts are partially inspired by Wozniak's <a href="https://en.wikipedia.org/wiki/SWEET16">SWEET16</a>, although that is a much lower-level VM than what I have in mind.]<br />
<br />
One thing I have aimed for is a rich set of addressing modes, important for raising the level of abstraction and for reasonable code density. Most 8-bit VMs that I've seen, like SWEET16, provide a rather low-level virtual machine typically requiring several instructions to do anything useful. Ugh!<br />
<br />
<h3>
Outline of the Virtual Machine</h3>
The proposed VM works in 16-bit words.<br />
<br />
A variable is any address in RAM.<br />
<br />
Op-codes take 0, 1, or 2 arguments (<i>if</i> op-codes take a third argument which is a fixed address to branch to). Typically, the first argument will be an assignment target.<br />
<br />
There are two "registers", <i>X </i>and <i>Y</i>, to hold the arguments for op-codes.<br />
<br />
The registers are set via the following instructions (the explanation uses C syntax for the semantics, recall that <i>& </i>in C is the "address-of" operator).<br />
<br />
<ul>
<li><i>Xk</i> -- load <i>X </i>with a constant; <i>X := &k.</i></li>
<li><i>Xv </i>-- load <i>X</i> with a variable; <i>X := &v.</i></li>
<li><i>X_k </i>-- subscript <i>X </i>by a constant; <i>X := &(X[k]).</i></li>
<li><i>X_v </i>-- subscript <i>X </i>by a variable; <i>X := &(X[v]).</i></li>
<li><i>X@</i> -- indirection through <i>X </i>(equivalent to <i>X_k 0</i>); <i>X := *X.</i></li>
</ul>
<div>
and similarly for the Y register.</div>
<div>
<br /></div>
<div>
Together, these allow for quite rich argument addressing. For example, loading register <i>X</i> with <i>x.i.foo.bar</i> (or, in C syntax, <i>x[i]->foo->bar</i>) would be expressed in the VM as</div>
<div>
<br /></div>
<div>
<i> Xv x</i></div>
<div>
<i> X_v i</i></div>
<div>
<i> X_k foo</i></div>
<div>
<i> X_k bar</i></div>
<br />
<br />
In the VM, <i>SP</i> is a global stack-pointer variable used by the <i>push, pop, call, </i>and <i>ret</i> op-codes.<br />
<div>
<br /></div>
<div>
Op-codes:</div>
<div>
<ul>
<li><i>X := Y; X += Y; X -= Y; X *= Y; X /= Y; X &= Y; X |= Y; X ^= Y; </i>etc.</li>
<li><i>inc X; dec X</i></li>
<li><i>push X</i> -- equivalent to<i> SP -= 2; </i><i>@SP := X</i>.</li>
<li><i>pop X</i> -- equivalent to X<i> := @SP; SP += 2</i>.</li>
<li><i>jp X</i> -- jump to address <i>X</i>.</li>
<li><i>call X</i> -- equivalent to <i>push IP; jp X</i> where <i>IP</i> is the address of the next op-code.</li>
<li><i>ret</i> -- equivalent to <i>pop IP.</i></li>
<li><i>poke X, Y</i> -- write the least significant byte of <i>Y </i>to address <i>X</i>.</li>
<li><i>X :=peek Y</i> -- assign the byte at address <i>Y </i>to<i> X</i>.</li>
<li><i>if X = Y A; if <i>X != Y A;</i> if <i>X < Y A;</i></i>etc. -- branch to address <i>A</i> if the test <i>fails</i> (i.e., <i>unless X = Y; X != Y; X < Y; </i>etc.).</li>
<li>And anything else that might be useful.</li>
</ul>
<div>
The odd inversion of the <i>if</i> op-codes is to support some sensible high-level syntax, such as</div>
</div>
<div>
<br /></div>
<div>
<i>if q != 0 { p /= q }</i></div>
<div>
<i><br /></i></div>
<div>
which would have the obvious translation.</div>
<div>
<br /></div>
<h3>
Example Code: Searching a Binary Tree</h3>
<div>
I'm making up some high-level syntax here, fill in any gaps using your imagination.</div>
<div>
<i><br /></i></div>
<div>
<i>// Symbols for the field offsets in our binary tree vertices.</i></div>
<div>
<i>Value .equ 0</i></div>
<div>
<i>Left .equ 1</i></div>
<div>
<i>Right .equ 2</i></div>
<div>
<i><br /></i></div>
<div>
<i>// We assume t is the root of the binary tree and we are looking for the vertex containing x.</i></div>
<div>
<i>loop {</i></div>
<div>
<i> if t = 0 { ret } // We hit null, we failed to find x.</i></div>
<div>
<i> t := t.Value</i></div>
<div>
<i> if t = x { ret }</i></div>
<div>
<i> if t < x { t := t.Left } else { t := t.Right }</i></div>
<div>
<i>}</i></div>
<div>
<br />
That, I claim, is fairly neat. The equivalent Z80 or 6502 assembly code would be far less transparent.<br />
<br />
The above would be expanded in a simple "assembler" into something like this (where <i>X</i> or <i>Y</i> is syntactically unchanged from its previous use, no op-codes need be omitted for set-up):<br />
<br />
<i>_1:</i><br />
<i> Xv t; Yk 0; if= _2</i><br />
<i> ret</i><br />
<i>_2:</i><br />
<i> Yv t; Y_k Value; :=</i><br />
<i> Yv x; if= _3</i><br />
<i> ret</i><br />
<i>_3:</i><br />
<i> if< _4</i><br />
<i> Yv t; Y_k Left; :=</i><br />
<i> Xk _5; jp</i><br />
<i>_4:</i><br />
<i> Yv t; Y_k Right; :=</i><br />
<i>_5:</i><br />
<i> Xk _1; jp</i><br />
<h3>
The Byte-code and Interpreter</h3>
Now, as an ex-Sinclair kiddie, I'm used to the Z80, so I'm going to use that to inform my idea of how the byte code might be expressed.<br />
<br />
Rather than interpreting byte-codes, I'm simply going to propose a sequence of <i>[op code address][arg]</i> or just <i>[op code address]</i> 16-bit values as my byte code. That is, every op-code is represented by the address of the machine code implementing it.<br />
<br />
For example:<br />
<br />
<i>Xk:</i><br />
<i> ld (X), SP: pop HL: ret </i>; Because the Z80 SP now holds the address of the constant.<br />
<br />
<i>Xv:</i><br />
<i> pop HL: ld (X), HL: ret</i><br />
<i><br /></i>
<i>X_k:</i><br />
<i> ld HL, (X): pop DE; add HL, DE: add HL, DE: ld (X), HL: ret</i><br />
<i><br /></i>
<i>':=':</i><br />
<i> ld HL, (Y): ld DE, (X): ldi: ldi: ret</i><br />
<i><br /></i>
<i>'+=':</i><br />
<i> ld HL, (X): ld DE, (Y)</i><br />
<i> ld A, (DE): inc DE: add (HL): ld (HL), A: inc HL</i><br />
<i> ld A, (DE): ld B, (HL): adc A, B: ld (HL), A</i> ; Ain't the Z80 horrible?<br />
<i> ret</i><br />
<br />
By pointing the Z80 stack-pointer at the byte-code stream, I can start executing it just by issuing the <i>ret</i> assembly instruction (which pops the next address off the stack and jumps to it).<br />
<br />
Going by this, each register set-up op-code and argument takes 4 bytes, each <i>if</i> op-code and jump address takes 4 bytes, and every other op-code takes 2 bytes.<br />
<br />
Under this scheme, the above binary tree search function would take 140 bytes. This is comparable to the equivalent Z80 machine code. It would run fairly quickly, the "interpretation" overhead being about as small as I can think of making it.<br />
<br />
<h3>
A Quick Comparison with SWEET16</h3>
This article on <a href="http://www.6502.org/source/interpreters/sweet16.htm#Opcode_Subroutines_">SWEET16</a> provides an 18-byte SWEET16 memory copy routine. How would our putative VM do?<br />
<br />
<i>loop {</i><br />
<i> if n = 0 { ret }</i><br />
<i> poke dest, src</i><br />
<i> inc dest</i><br />
<i> inc src</i><br />
<i>}</i><br />
<i><br /></i>
which would turn into the following byte code:<br />
<br />
<i>_1:</i><br />
<i> Xv n; Yk 0; if= _2</i><br />
<i> ret</i><br />
<i>_2:</i><br />
<i> Xv dest; Yv src; poke</i><br />
<i> inc</i><br />
<i> Xv src; inc</i><br />
<i> jp _1</i><br />
<i><br /></i>
which, by my reckoning, comes to 36 bytes, twice what SWEET16 requires. Not such a big deal given how much more comfortable the new VM feels to program in than SWEET16.<br />
<br />
<h3>
Closing Thoughts</h3>
I wish I'd thought of this back in the early 1980s! While coding everything in assembly language was fun, it was error prone, and the weirdness of the Z80 turned every little design decision into a mental challenge. I think this VM (and accompanying compiler) would have been possible to write in BASIC and a handful of assembler in a few evenings and that it would been a great tool for bootstrapping other projects.<br />
<br />
(Later in life I ended up spending fifteen years in programming language design and compiler research...)<br />
<br />
<br /></div>
Rafehttp://www.blogger.com/profile/10169325688704651715noreply@blogger.com1tag:blogger.com,1999:blog-6767707.post-65978313646950285122017-09-15T11:27:00.002+09:302017-11-16T12:04:24.177+10:30Paul Graham's "What you can't say"This is well worth reading:<br />
<br />
<a href="http://www.paulgraham.com/say.html">http://www.paulgraham.com/say.html</a>Rafehttp://www.blogger.com/profile/10169325688704651715noreply@blogger.com1tag:blogger.com,1999:blog-6767707.post-91623800353273757062017-08-29T09:05:00.000+09:302017-08-29T09:05:42.400+09:30Renewables, sigh... https://wattsupwiththat.com/2017/08/28/wind-power-some-basic-facts/Rafehttp://www.blogger.com/profile/10169325688704651715noreply@blogger.com0tag:blogger.com,1999:blog-6767707.post-39096286296301466682016-05-03T14:00:00.002+09:302016-05-03T14:00:30.475+09:30Od -- it's like Mithril and Knockout, but betterI've loved using <a href="http://knockoutjs.com/">Knockout </a>these last few years, but struggled with its unpredictable performance. I really like the idea of <a href="http://mithril.js.org/">Mithril</a> (and the other virtual-DOM schemes), but it has really wonky problems if you move a component around in your DOM.<br />
<br />
To address these issues, I recently came up with my own scheme, <i>Od</i>,<i> </i>and I really like it:<br />
<br />
<a href="https://github.com/ralphbecket/Web/tree/master/Od">https://github.com/ralphbecket/Web/tree/master/Od</a><br />
<br />
The web site there has plenty of explanation, examples, and documentation.Rafehttp://www.blogger.com/profile/10169325688704651715noreply@blogger.com0tag:blogger.com,1999:blog-6767707.post-77889516773006644512016-05-03T13:50:00.002+09:302016-05-03T13:56:03.296+09:30TypeScript, the beautiful face of the ugly tapeworm that is JavaScript. It's the only way to do this stuff.<br />
<br />
To add a TypeScript component to a non-TypeScript Visual Studio project, add the following to the end of the project <i>.csproj</i> file (do this in <i>notepad </i>or somesuch; adjust the TypeScript path version accordingly):<br />
<br />
<script src="https://gist.github.com/ralphbecket/edd35d0c7c73bbae485c2c207e2aaaf8.js"></script>
<div>
Reload the project and away you go. When you add TypeScript files to your project, you can set their <i>BuildAction</i> properties to <i>TypeScriptCompile,</i> if this doesn't happen automatically.</div>
Rafehttp://www.blogger.com/profile/10169325688704651715noreply@blogger.com0tag:blogger.com,1999:blog-6767707.post-41860838441271363402016-03-18T14:18:00.001+10:302016-03-18T14:18:12.689+10:30On Python (vulgar, but funny)https://semitwist.com/articles/article/view/why-i-hate-python-or-any-dynamic-language-reallyRafehttp://www.blogger.com/profile/10169325688704651715noreply@blogger.com0tag:blogger.com,1999:blog-6767707.post-59764163307326680562016-03-14T09:53:00.001+10:302016-03-14T09:53:03.732+10:30OO is EmbarrassingRight on: Brian Will on <a href="https://www.youtube.com/watch?v=IRTfhkiAqPw&app=desktop">Object-Oriented Programming is Embarrassing: 4 Short Examples</a>.<br />
<br />
<br />
<br />Rafehttp://www.blogger.com/profile/10169325688704651715noreply@blogger.com2tag:blogger.com,1999:blog-6767707.post-32523338043344468092016-02-17T09:13:00.001+10:302016-02-17T09:13:25.333+10:30Wonderful explanation of quantum weirdnessA discussion on the excellent <a href="http://quora.com/">Quora</a> site led me to this <a href="http://lesswrong.com/lw/pd/configurations_and_amplitude/">series of articles</a> explaining the mysteries of quantum mechanics at a level comprehensible by an armchair physicist or even myself. No hand-waving, nothing above some basic arithmetic with complex numbers, perspicuous and concise.<br />
<br />
While on the topic, I'll take the opportunity to plug <i><a href="http://www.amazon.com/gp/product/B00KFEK0I8">Introducing Quantum Theory</a></i> which tells the tale of how physics moved from classical to quantum theory. There's no mathematics in the book, instead it tells the story of all the things that didn't quite make sense along the way and how Dirac, Schroedinger, Einstein, and the rest of the gang arrived at the new model. Don't be put off by the cartoonesque presentation: it is a book for grown-ups. I've bought it three times so far, having loaned it out twice and not had it returned...Rafehttp://www.blogger.com/profile/10169325688704651715noreply@blogger.com0tag:blogger.com,1999:blog-6767707.post-18979484587047591412015-09-30T21:51:00.001+09:302015-10-21T22:23:22.660+10:30Faster Sprites on the ZX Spectrum<p>
This is one for anybody who grew up programming the Sinclair ZX Spectrum, a
wonderful machine from the early '80s microcomputer days, sporting a 3.5 MHz
Z80 CPU and 48 KBytes of RAM.
</p>
<p>
Lacking hardware sprites, ZX Spectrum programmers were/are forced to draw
directly to the display. Moving a sprite therefore involves removing the
previous sprite image before drawing the new one. Since sprites can overlap,
this typically means first removing all previously drawn sprites before
redrawing any of them in their new positions (unless you use XOR to draw your
sprites, in which case removing a sprite is just redrawing it in its previous
position and removing/redrawing can occur in any order).
</p>
<p>
My code implementing the ideas here is available at
<a href="https://github.com/ralphbecket/Z80/tree/master/FasterSprites">
https://github.com/ralphbecket/Z80/tree/master/FasterSprites
</a>
</p>
<h3>Screen Layout and Timing</h3>
<p>
To achieve flicker-free animation, all drawing must occur before the display
refresh crosses the part of the screen being updated. This means
understanding something of the ZX Spectrum display arrangement:
</p>
<p>
<pre style="font-family: courier; font-size: x-small; line-height: 1;">
+-----------------------------------------------------+
| Border: 64 scan lines before display file. |
| |
| +-------------------------------------------+ |
| | | | | | | Display: 256 x 192 pixels| |
| |--+--+--+--+--+-- arranged as 32 cols of | |
| | | | | | | 24 rows of 8x8 cells. | |
| |--+--+--+--+--+-- | |
| | | | | | | | |
| |--+--+--+--+--+-- | |
| | | | | | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| +-------------------------------------------+ |
| |
| |
+-----------------------------------------------------+
</pre>
</p>
<p>
The border is a single, block colour surrounding the display file. It starts
64 scan lines before the display file. The display file is a 256 x 192 pixel
bitmap arranged as 24 rows of 32 columns of 8x8 pixel cells. A separate
attribute map specifies a single foreground/background colour pair for each
cell (i.e., colours can only be specified at the level of cells, not pixels).
</p>
<p>
Each line takes 224 Ts for the hardware to display, where 1 T = 1 t-state = 1
clock tick of the 3.5 MHz Z80 CPU. The only reliable way to know when to start
drawing is to wait for the vertical refresh interrupt, which occurs every time
the hardware starts drawing a new frame from the top of the screen. To put
this in perspective, a typical Z80 instruction takes around 4 to 16 Ts to
execute.
</p>
<p>
Assuming we want flicker-free animation, this means we have 224 * 64 = 14,336
Ts from the start of the interrupt before the display refresh catches up with
us. (There is a clever trick involving waiting for the display hardware to
just pass the top of the display file and start drawing behind it, but it isn't
very portable.) A whole frame takes 224 * (64 + 192 + 64) = 71,680 Ts.
</p>
<h3>Display Addressing</h3>
<p>
The display file is arranged as 32 columns of 24 rows of 8x8 pixel cells.
Each cell has a colour attribute and eight bytes of bitmap data. Bitmap
addresses are given in binary as 010RRLLL RRRCCCCC where RRRRR is the row
number (0..23), CCCCC is the column number (0..31), and LLL is the line number
within the cell (0..7). Attribute addresses are given as 010110RR RRRCCCCC.
The curious looking bitmap addressing is cunningly arranged so that successive
lines within a cell can be reached by simply incrementing the upper byte of the
address.
</p>
<h3>Moving Bytes to the Display</h3>
<p>
Now, how best to move bits on to the display file? Setting aside the attribute
file for now, there are a number of options. For the sake of this part of the
discussion, I'm going to assume we are just interested in moving bytes from
some (pre-shifted) source to a cell on the display file.
</p>
<p>
Here are the most common approaches, assuming the source bitmap address is in
DE or SP and the target display bitmap cell address is in HL:
</p>
<p>
<b>(1) Naive copy</b>
<pre>
ld A, (DE) ; 7 Ts
ld (HL), A ; 7 Ts
inc DE ; 6 Ts
inc H ; 4 Ts
; ... and so on for the next 7 lines.
; Cost per byte: 24 Ts
</pre>
</p>
<p>
<b>(2) Loading from the stack</b>
<pre>
pop DE ; 10 Ts
ld (HL), D ; 7 Ts
inc H ; 4 Ts
ld (HL), E ; 7 Ts
inc H ; 4 Ts
; ... and so on for the next 3 line pairs.
; Cost per byte: 16 Ts
</pre>
</p>
<p>
<b>(3) Self-modifying code</b>
<pre>
ld (HL), line_1 ; 10 Ts
inc H ; 4 Ts
ld (HL), line_2 ; 10 Ts
inc H ; 4 Ts
; and so on for the next 6 lines.
; Cost per byte: 14 Ts
</pre>
</p>
<p>
<b>(4) Copying from double buffer with some SMC</b>
<pre>
ld SP, src_addr
pop HL
pop DE
pop BC
; ... and so on for other register pairs.
ld SP, disp_addr
; ...
push BC
push DE
push HL
; and repeat until done.
; Cost per byte: 10.5 Ts
</pre>
</p>
<p>
<h3>Comparing the Options</h3>
</p>
<p>
We can safely discard option (1) as the slowest and having no compensating
merits.
</p>
<p>
Option (4) essentially assumes we have written to some backing buffer and are
simply copying it to the display. A full screen takes 192 * 32 * 10.5 = 64,512
Ts, which, being approximately the same as the 224 * (64 + 192 + 64) = 71,680
Ts it takes for the hardware to display a full frame, leaves no time for
drawing the backing buffer in the first place. In other words, option (4) is
only acceptable if we are prepared to render every other frame or less. For
this reason, I'm going to discard option (4) -- I want to see what we can
manage in the opening 14,336 Ts of every frame.
</p>
<p>
Options (2) and (3) remain. In theory, we could manage 14,336 / (8 * 16) = 112
cells per frame with option (2) or 14,336 / (8 * 14) = 128 cells per frame with
option (3).
</p>
<p>
These figures, of course, are without counting bookkeeping such as setting up
DE/SP/HL or writing colour attributes. For example, for each cell we'd need
something like the following to set up the registers and set the attribute
byte:
</p>
<pre>
pop HL ; 10 Ts -- load the attr addr.
pop DE ; 10 Ts -- load the attr byte and disp upper addr.
ld (HL), D ; 7 Ts -- write the attr byte.
ld H, E ; 4 Ts -- convert attr addr to disp addr.
</pre>
<p>
Let's assume we need at least 40 Ts of
overhead per cell or 40 / 8 = 5 Ts per byte. This gives 14,336 / (8 * 21) = 85
cells per frame for option (2) and 14,336 / (8 * 19) = 94 cells per frame for
option (3).
</p>
<h3>How many sprites at full tilt?</h3>
<p>
Assuming a "standard" 16 x 16 pixel sprite, pre-shifted to any position, such a
sprite will occupy a 3 * 3 cell region. Therefore, option (2) allows for
85 / 9 = 9.4 sprites per frame, while option (3) allows for 94 / 9 = 10.4
sprites per frame.
</p>
<h3>Memory budgets</h3>
<p>
The downside of option (3) is the amount of memory it requires. Each cell that
we draw requires the following space for self-modifying code:
</p>
<pre>
ld HL, attr_addr ; 3 bytes
ld (HL), attr ; 2 bytes
ld H, disp_hi_addr ; 2 bytes
ld (HL), line_1 ; 2 bytes
inc H ; 1 byte
; ... last two lines repeated eight times in total.
; Total cost: 30 bytes.
</pre>
<p>
Option (2) requires less space per cell, just the data for the drawing loop:
</p>
<pre>
dw attr_addr
db attr, disp_hi_addr
db line_1, line_2, ..., line_8
; Total cost: 12 bytes.
</pre>
<h3>The winner is...</h3>
<p>
After trying both approaches, I found option (3) to be costly in memory and
costly to set up each frame, leaving little time for game logic. For that
reason, I claim option (2) is the optimal strategy <em>provided</em> we are
aiming for full-screen full-speed flickerless animation <em>without</em>
resorting to beam chasing tricks.
</p>
<h3>Digression: full cells or part cells</h3>
<p>
The discussion so far has assumed we are always drawing full cells. Consider a
16x16 sprite, this, when shifted into position, will occupy 3x16 bytes.
However, it will (seven times out of eight) cross three cell rows. We can
either optimise drawing of complete cells (3x24 bytes) or we can try to draw
only what we need to (3x16 bytes). There are two problems with the latter
approach. The first is what to do when sprites overlap (see below). With the
prepare-first-then-draw-quickly strategy of option (2), we can handle the
merging of overlapping images in the preparation period, which is outside the
time-sensitive drawing period. With the just-draw-what-you-need-and-no-more
strategy, it's not at all clear how you could manage this. You <em>could</em>
restrict yourself to just xor-drawing and add the xor-code inline, but that
costs you -- here's the code for doing that for one byte pair:
</p>
<pre>
; Xor-drawing.
pop DE ; 10 Ts - Fetch two lines
ld A, (HL) ; 7 Ts
xor D ; 4 Ts
ld (HL), A ; 7 Ts
inc H ; 4 Ts
ld A, (HL) ; 7 Ts
xor E ; 4 Ts
ld (HL), A ; 7 Ts
inc H ; 4 Ts
; All repeated four times...
; Total cost: 27 Ts per byte
</pre>
<p>
Add in the 5 Ts cost for attribute handling et cetera and we're up to 32 Ts per
byte, limiting us to 14,336 / (8 * 32) = 56 cells. Not nearly so good compared
to the 85 cells we can draw with option (2). Now, you might say, "but in this
case we're only going to draw just what we need, so our 16x16 sprites only
occupy 6 cells worth of space, giving us 56 / 6 = 9 sprites per frame". That
is true, until you consider that each of those sprites will, seven times out of
eight, cross two cell row boundaries. You now need to add logic to handle both
"drawing from one to eight lines of a cell (top or bottom)" and "updating the
display address for the top of the next cell row". Neither of those operations
are cheap -- I've tried every trick I can think of -- resulting in a more
complex, slower solution. That's why I've decided to stick to drawing complete
cells: the overheads involved in drawing partial cells are simply too high to
pay off.
</p>
<h3>Overlapping sprites</h3>
<p>
When cells from two sprites overlap, we need to decide how to handle the
situation. The simplest approach of "one of them wins, the other is never
seen" looks awful. Instead, we want to combine them somehow -- typically
using or- or xor-merging or even using masking (and-mask then or-image) if
we want to be really fancy. The key thing is, if we want to sustain nine
sprites per frame, this "compositing" has to occur in the set-up period, not in
the drawing period.
</p>
<p>
One way to do this is to maintain a map, much like the display attribute map,
indicating which of the cells on screen is to be drawn in the next frame. Our
"used cells" map will, instead, hold zero for unused cells, and positive number
n if the nth "draw list" record holds the data (attribute pointer, attribute,
display pointer high byte, and eight bitmap bytes) for that cell.
</p>
<pre style="font-family: courier; font-size: x-small; line-height: 1;">
Used cell map:
+-------------------------------------+
| | | | | | |
|--+--+--+--+--+-- |
| | 1| 4| 7| | |
|--+--+--+--+--+-- |
| | 2| 5| 8|11|14 |
|--+--+--+--+--+-- |
| | 3| 6| 9|12|15 |
|--+--+--+--+--+-- |
| | | |10|13|16 etc. |
| |
| |
+-------------------------------------+
Draw list:
1 : attr ptr, attr, disp hi ptr, bits1 ... bits 8
2 : attr ptr, attr, disp hi ptr, bits1 ... bits 8
3 : attr ptr, attr, disp hi ptr, bits1 ... bits 8
4 : attr ptr, attr, disp hi ptr, bits1 ... bits 8
etc.
</pre>
<p>
When we want to add a cell to our draw list, we first examine the corresponding
entry in the used cells map. If this is a zero, then we need to set up a new
draw list record for this cell and update the used cells map accordingly. If
the used cells map contains a non-zero value, that tells us that the cell
already has a draw list record and which record we must adjust in our
compositing operation.
</p>
<h3>Erasing sprites</h3>
<p>
The Speccy allows for a neat, cost effective erasure trick provided you don't
have to preserve any background. Simply set the attributes for any cell to be
erased to have the same foreground and background colours! No bitmap data
needs to be changed at all. This is a single byte write and vastly cheaper
than redrawing using xor or any similar shenanigans.
</p>
<p>
Thankfully, our used cell map provides a convenient mechanism for working out
which cells need to be erased from the previous frame (we don't want to blank
any cells we are drawing to this frame, just the ones that were drawn to in
the previous frame, but not this frame).
</p>
<p>
Basically, we need to track which cells were used in the previous frame.
After drawing the current frame, we see which cells were used in the previous
frame, but not in this frame (i.e., we check a list of cells from the
previous frame against the cells used in this frame -- via the used cells
map -- and write the blank attribute value to those cells).
</p>
<h3>Empirical testing</h3>
<p>
Using Simon Brattel's marvellous Zeus assembler/emulator/debugger/profiler
I have collected some performance data. Running nine 16x16 pixel (i.e., 3x3
cell) sprites I observe the following T-states per frame:
<ul>
<li> 13,054 Ts spent drawing cells (tidily within our 14,336 T budget);</li>
<li> 8,087 Ts spent blanking cells (over our 14,336 T budget);</li>
<li> 36,587 Ts spent setting up cells;</li>
<li> 9,801 Ts spent managing sprites.</li>
</ul>
This is rather encouraging. I am somewhat surprised by the amount of time
needed for blanking, but there's a fair amount of other bookkeeping going
on in there and only a few cells typically need blanking per frame (sprites
will typically move less than one cell width per frame). There is plenty of
time left over for game logic, sound, and what-have-you.
</p>
<h3>Further thoughts</h3>
<p>
Given that so much remaining time is available per frame, it seems quite
likely that a similar number of masked sprites could be supported at full
frame rate. This would require adjustment to the "compositing" code and
sprite layout, but nothing major. If there were a non-blank background
to deal with, that would also complicate matters somewhat, but I can
imagine at least a couple of plausible approaches.
</p>
<p>
All that said, I'm happy that my analysis played out in practice. I'm also
happy that this is a shape-agnostic solution: any sprite sizes can
be accommodated provided they fit within the approximately 100 cells per
frame. That would be one huge 72x72 pixel sprite or four 32x32 pixel sprites
or any other combination. All at 50Hz, in full colour, with compositing.
</p>
Rafehttp://www.blogger.com/profile/10169325688704651715noreply@blogger.com8tag:blogger.com,1999:blog-6767707.post-13072825632359112502015-09-04T09:39:00.001+09:302015-09-27T10:04:13.389+09:30Simulating any biased coin with a fair coinI recently discovered this cutie on the web.<br />
<ol>
<li>Using a fair coin, you can simulate a coin with any desired bias.</li>
<li>The <i>expected </i>number of tosses you need to do this is just two!</li>
</ol>
<div>
Preliminaries: let's count heads as 1, tails as 0, and we'll write the desired probability of heads in our simulated biased coin as the binary fraction `p = 0.p_1p_2p_3...` where `p_i` is the `i`th binary digit of the probability `p`.</div>
<div>
<br /></div>
<div>
<b>Procedure: </b>keep tossing the fair coin until we get heads, on the `k`th toss. Return `p_k` as the result and stop.</div>
<div>
<br /></div>
<div>
<b>Proof this works: </b>the probability of seeing a "head" (i.e., a 1) in this process is `P(head) = sum_(1<=k)p_k/2^k` which is exactly `p` by definition.<br />
<br />
<b>Expected number of tosses: </b>the expected number of tosses is `E(k) = sum_(1<=k)k/2^k`. Now, we can rewrite this two different ways. One way, we can do a "change of variable" from `k` to `k+1`, giving `E(k) = sum_(0<=k)(k+1)/2^(k+1)`. Let's call this form, `A`. Another way, we can take the original equation and add a harmless zero term to the front of the sum, giving `E(k) = sum_(0<=k)k/2^k`. Let's call this form, `B`. Now, since `E(k) = A = B` we must have<br />
`E(k) = 2A - B`<br />
`qquad = 2sum_(0<=k)(k+1)/2^(k+1) - sum_(0<=k)k/2^k`<br />
`qquad = sum_(0<=k)(k+1)/2^k - sum_(0<=k)k/2^k`<br />
`qquad = sum_(0<=k)(k + 1 - k)/2^k`<br />
`qquad = sum_(0<=k)1/2^k`<br />
`qquad = 2`.<br />
<br />
I find it delightful that the expected number of tosses is independent of the target probability!</div>
Rafehttp://www.blogger.com/profile/10169325688704651715noreply@blogger.com0tag:blogger.com,1999:blog-6767707.post-62206698154556121122014-12-14T15:33:00.001+10:302014-12-14T15:33:11.593+10:30Rawb - efficient Knockout-friendly web controlsI've released Rawb (http://rawb.codeplex.com), a library of efficient web controls for use with Knockout (http://www.knockoutjs.com). The grid control is a particularly nice piece of work, in my opinion.Rafehttp://www.blogger.com/profile/10169325688704651715noreply@blogger.com2tag:blogger.com,1999:blog-6767707.post-69080137695209275282014-12-14T15:31:00.000+10:302014-12-14T15:31:20.678+10:30Spanner releasedI thought I'd posted this months ago! Oh, well: Spanner is now available for download from http://spanner.codeplex.com . A tutorial site with documentation can be found at http://www.spannerjs.com .
In case you have forgotten, Spanner is my attempt to provide an integrated scheme for writing web applications as embedded domain-specific languages in C#. It's rather nice, if I say so myself.Rafehttp://www.blogger.com/profile/10169325688704651715noreply@blogger.com0tag:blogger.com,1999:blog-6767707.post-78250451777307825322014-02-10T10:32:00.000+10:302014-02-10T10:32:28.563+10:30Ahh, JavaScript, how I loathe thee.Okay, to be fair, this complaint isn't really about JavaScript so much as the JavaScript/DOM interface. Specifically, the ordering of event handlers.
<br />
<br />
Whenever one finds oneself struggling with interacting event handlers, the order of which is not consistent across browsers, one ends up doing something horrid like this:
<br />
<br />
<pre> myEventHandler = function (evt) {
setTimeout(function () {
... my intended side-effect ...
},
100 // Magic delay in milliseconds,
// which we pray will force the
// right order of events.
};
</pre>
<br />
Bleh. A thousand times, bleh. Please write in if you have a good solution to this problem.Rafehttp://www.blogger.com/profile/10169325688704651715noreply@blogger.com1tag:blogger.com,1999:blog-6767707.post-80107699426616802013-10-29T09:19:00.000+10:302013-10-29T09:19:28.429+10:30How to find the parameter names of a C# lambaLast night I tweaked <a href="http://spanner.codeplex.com">Spanner</a> so function parameter names in the generated JavaScript match the parameter names of the C# <i>Func</i> from which they were generated.
I had a horrible feeling I'd have to stoop to using and parsing <i>Expression</i> values, but it turns out that isn't necessary at all. Behold, the answer:
<script type="syntaxhighlighter" class="brush: csharp">
string[] funcParamNames(Delegate f) {
return f.Method.GetParameters().Select(x => x.Name).ToArray();
}
</script>
And...
<script type="syntaxhighlighter" class="brush: csharp">
Func<int, bool, string, int> f = (x, y, z) => 69;
...
funcParamNames(f) === string[] {"x", "y", "z"}
</script>
Rafehttp://www.blogger.com/profile/10169325688704651715noreply@blogger.com0tag:blogger.com,1999:blog-6767707.post-21803301462635112122013-10-14T20:32:00.000+10:302013-10-14T20:33:16.059+10:30Spanner is released!I have, this day, made <a href="https://spanner.codeplex.com/">Spanner </a>available on <i>Codeplex</i>.
It's an alpha release, which means I intend to make minor changes to the design and major improvements to the documentation, add tutorials, and all that.
The <i>ToDoMVC</i> example is temporarily broken. Fixing that is first on my (real) to-do list -- as my old PhD supervisor used to say to me, beware of showing an idiot something half finished.Rafehttp://www.blogger.com/profile/10169325688704651715noreply@blogger.com0tag:blogger.com,1999:blog-6767707.post-1066282666967037402013-09-20T20:30:00.002+09:302013-09-20T20:59:16.537+09:30A taste of SpannerSpanner is the name of my soon-to-be-released open source statically typed domain specific language for building <a href="http://knockoutjs.com/">Knockout</a> web applications in C#.
Here is a very brief sample, which I will let stand by itself:
<script type="syntaxhighlighter" class="brush: csharp">
var x = ObsVar<int>(0);
var y = ObsVar<int>(0);
var z = ComputedVar<int>(x + y);
var evenOrOdd = ComputedVar(Cond<string>(z % 2 == 0, "even", "odd"));
var html = StandardSpannerHtmlPage("Spanner sample",
Div(
Input().KoValue(x), " + ", Input().KoValue(y), " = ", Show(z),
" which is ", Show(evenOrOdd), "."
)
);
var webPage = WebPage("SpannerSample", html, Noop());
var ctxt = new Context(@"C:\Tmp", singlePage: true);
ctxt.WriteWebPage(webPage);
</script>
Build and run this program and you get the following web page:
<hr>
<div>
<input data-bind='value: _x1'>
+
<input data-bind='value: _x2'>
=
<span data-bind='text: _x3()'>
</span>
which is
<span data-bind='text: _x4()'>
</span>
.
</div>
<script src='http://cdnjs.cloudflare.com/ajax/libs/knockout/2.3.0/knockout-min.js'>
</script>
<script>
// This is the Spanner runtime environment.
// NOTE: This runtime assumes Knockout is available (http://www.knockoutjs.com).
//
var spanner = new (function () {
// ---- Shim code. ----
// From https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/forEach
// Production steps of ECMA-262, Edition 5, 15.4.4.18
// Reference: http://es5.github.com/#x15.4.4.18
if (!Array.prototype.forEach) {
Array.prototype.forEach = function forEach(callback, thisArg) {
'use strict';
var T, k;
if (this == null) {
throw new TypeError("this is null or not defined");
}
var kValue,
// 1. Let O be the result of calling ToObject passing the |this| value as the argument.
O = Object(this),
// 2. Let lenValue be the result of calling the Get internal method of O with the argument "length".
// 3. Let len be ToUint32(lenValue).
len = O.length >>> 0; // Hack to convert O.length to a UInt32
// 4. If IsCallable(callback) is false, throw a TypeError exception.
// See: http://es5.github.com/#x9.11
if ({}.toString.call(callback) !== "[object Function]") {
throw new TypeError(callback + " is not a function");
}
// 5. If thisArg was supplied, let T be thisArg; else let T be undefined.
if (arguments.length >= 2) {
T = thisArg;
}
// 6. Let k be 0
k = 0;
// 7. Repeat, while k < len
while (k < len) {
// a. Let Pk be ToString(k).
// This is implicit for LHS operands of the in operator
// b. Let kPresent be the result of calling the HasProperty internal method of O with argument Pk.
// This step can be combined with c
// c. If kPresent is true, then
if (k in O) {
// i. Let kValue be the result of calling the Get internal method of O with argument Pk.
kValue = O[k];
// ii. Call the Call internal method of callback with T as the this value and
// argument list containing kValue, k, and O.
callback.call(T, kValue, k, O);
}
// d. Increase k by 1.
k++;
}
// 8. return undefined
};
}
// ---- Spanner code below. ----
var self = this;
// Set the ith item of an observable array.
//
self.setObsIth = function (obs, i, x) {
var xs = obs();
xs[i] = x;
obs(xs);
};
// Set a field of an observable object.
//
self.setObsField = function (obs, setF, x) {
var obj = obs();
setF(obj, x);
obs(obj);
};
// Push an item onto an array held in an observable.
// Since this is an assignment, if the underlying
// array is shared by other observables, they will
// be updated in value, but Knockout will not know of
// it and your web page will behave unpredictably.
//
// This is characteristic of all operations here with side effects.
//
self.pushObsArray = function (obs, x) {
var xs = obs();
xs.push(x);
obs(xs);
};
self.popObsArray = function (obs) {
var xs = obs();
var x = xs.pop();
obs(xs);
return x;
};
self.reverseObsArray = function (obs) {
var xs = obs();
xs.reverse();
obs(xs);
};
self.shiftObsArray = function (obs) {
var xs = obs();
var x = xs.shift();
obs(xs);
return x;
};
self.sortObsArray = function (obs, f) {
var xs = obs();
var x = xs.sort(f);
obs(xs);
};
self.spliceObsArray = function (obs, begin, howMany, ys) {
var xs = obs();
xs = [].splice.apply(xs, [begin, howMany].concat(ys));
obs(xs);
};
self.unshiftObsArray = function (obs, ys) {
var xs = obs();
xs = [].unshift.apply(xs, ys);
obs(xs);
};
self.replaceObsString = function (obs, pattern, replacement, flags) {
var x = obs();
x.replace(pattern, replacement, flags);
obs(x);
};
// JavaScript's "(almost) any type can be extended with properties" scheme doesn't fit
// C#, so in those cases we need to write "type adaptors".
self.regexExec = function (re, str) {
var result = re.exec(str);
if (!result) return result;
var matchedSubstr = result[0];
var parenMatches = [];
for (var i = 1; i < result.length; i++) parenMatches.push(result[i]);
// This return value must match class UI.RegexMatch.
return {
MatchedSubstr: matchedSubstr,
ParenMatches: parenMatches,
MatchIndex: result.index,
MatchInput: result.input
};
}
// Set up Knockout extensions for our primitive number types.
// We need to ensure these are assigned numeric values when
// receiving input from an input field.
// An obsInt is always an integer or NaN.
// String assignments are coerced to numbers.
//
self.obsInt = function (initialValue) {
var obs = ko.observable(initialValue);
var obsInt = ko.computed({
read: obs,
write: function (x) {
var num = +x;
obs(num.toString() === num.toFixed() ? num : NaN);
}
});
obsInt(obs());
return obsInt;
};
// An obsDouble is always a number or NaN.
// String assignments are coerced to numbers.
//
self.obsDouble = function (initialValue) {
var obs = ko.observable(initialValue);
var obsDouble = ko.computed({
read: obs,
write: function (x) {
var num = +x;
obs(num);
}
});
obsDouble(obs());
return obsDouble;
};
self.windowOnLoad = function (f) {
window.onload = (function (wol) {
if (wol) wol();
if (f) f();
})(window.onload);
}
// All done.
return self;
})();
function SpannerSample() {
var self = this;
self._x1 = spanner.obsInt(0);
self._x2 = spanner.obsInt(0);
self._x3 = ko.computed(function () { return (self._x1() + self._x2()); });
self._x4 = ko.computed(function () { return (((self._x3() % 2) === 0) ? "even" : "odd"); });
{} /* No op. */;
};
spanner.windowOnLoad(function () { ko.applyBindings(new SpannerSample()); });
</script>
<hr>
Now, this is a fairly trivial example, but it illustrates some of the key points about Spanner.
<ul>
<li> A Spanner program is completely, strongly, statically typed -- it's just C#. As soon as you make a type error in your program,
Visual Studio will list the problem areas in the error window. The generated JavaScript code cannot contain
type errors (well, up to the point where you start including third-party JavaScript libraries). This alone makes Spanner valuable.
<li> Strong typing means you get strong refactoring support from Visual Studio.
<li> You don't need special syntax for expressions: you can freely use +, -, /, *, etc.
<li> Because you are writing in C#, you have all the usual mechanisms for abstraction (i.e., functions!). You are not
limited to the ad hoc hodge podge of compromises necessary in ordinary HTML + JavaScript development.
<li> You don't need special syntax for Knockout observables -- Spanner knows which variables are observables and which
are plain old JavaScript variables and generates the appropriate code for reading and writing them. Indeed, observables
are much more transparent in Spanner than they are in JavaScript.
<li> All the documented, well-typed parts of Knockout are supported.
<li> Every HTML element and every well-typed JavaScript construct is represented by a correspondingly named Spanner function.
<li> The correspondence between HTML attributes and view model variables is transparent -- you never need to embed code or
identifiers in HTML strings. This is another key benefit of Spanner.
<li> Spanner handles all the usual types (int, double, string, arrays, enums, POCOs). There are no special cases.
<li> Spanner supports modularisation via view/view-models pairs, "global" models (i.e., libraries), strongly typed templates,
and use of arbitrary third-party libraries.
<li> Spanner will sort all your code dependencies for you and emit modules and variable definitions in the right order.
<li> Spanner provides mechanism rather than policy. As long as you are happy using Knockout for your observables, it is
entirely up to you how to structure your web application.
<li> Spanner makes it easy to mix hand-written JavaScript with Spanner-generated JavaScript. The two worlds work quite
comfortably together, provided the non-Spanner JavaScript has a well-typed API (hah!).
<li> Building happens in the blink of an eye. If you are used to waiting ages for MVC to do its thing, Spanner's instant
turn-around will be a relief.
<li> Normal web development is excruciating for anyone used to more sophisticated languages. You spend so much of your working
day chasing down trivial bugs and the primitive languages involved practically force you to turn your code into a mess of
expedient short cuts. I wrote Spanner as a reaction, to see how far I could Do It Right while using a mainstream programming
language (trust me, if we all migrated to F# or Haskell, Spanner would look a lot less cunning).
<li> Using Spanner will clearly make you more attractive to members of your favourite sex.
</ul>
Watch this space. I have the code completed, tested, and documented. I am just putting together a web site with a tutorial and some
demonstrations (as my old PhD supervisor said to me, never show an idiot something half finished :-)). Being a family man
with a full time job, this may take a week or three...Rafehttp://www.blogger.com/profile/10169325688704651715noreply@blogger.com9tag:blogger.com,1999:blog-6767707.post-19035771999095006912013-09-19T13:59:00.004+09:302013-09-19T14:04:42.269+09:30Damn you, C# !For some time now I have been working on a pet project: a domain specific language implemented as a set of C# types intended to support the creation of complex single-page web applications, generating everything including the HTML, CSS, and JavaScript. I am more than a little pleased with the result and will be making it open source <i>any day now</i>. [My motivation for this is that I've spent the last two years immersed in the web world, which is clearly a brilliant idea implemented by psychopaths and idiots -- I will rant in more detail on this topic at some later point.]<br />
<br />
Speaking of points, back to the subject in hand.<br />
<br />
My DSL is, in large part, based around a type <i>Expr<T><t></t></i> denoting a JavaScript expression of type <i>T</i>. To make life convenient, I have added implicit casts from <i>string</i> to <i>Expr</i><i><string></i> (etc.) and overridden the standard operator set <i>{+, -, *, /, %, ==, !=, <, <=, etc.}</i>. So far so good. I can write expressions such as <i>x + (y * z)</i> in my DSL directly in C# where <i>x</i>,<i> y, </i>and <i>z </i>are <i>Expr</i><i><int></i> or <i>Expr</i><i><double></i><i><double>,</double></i> and so forth.<br />
<br />
There are two flies in the ointment.<br />
<br />
<b>Ointment fly number 1: you can't freely overload the indexing operator.</b><br />
<br />
I would like to be able to write <i>xs[i]</i> to denote the JavaScript expression for the <i>i</i>th element of the array <i>xs</i>. Tragically, I can't do this. Here's what the implementation might look like if you could:<br />
<br />
<i>public static Expr</i><i><T></i><i><t>this[Expr</t></i><i><int></i><i><t><int>i] { get { return ...; } }</int></t></i><br />
<i><br /></i>
But, noooo, I can't do this because this has to be defined in the <i>Expr</i><i><T></i> class and that means <i>this</i> has type <i>Expr</i><i><T></i> rather than <i>Expr</i><i><T[]></i>, which is what is required. Instead I have to provide some awkward named function, <i>Ith(xs, i)</i> to do the job. Ugh.<br />
<br />
Damn you, C# !<br />
<br />
<b>Ointment fly number 2: you can't freely overload the << operator.</b><br />
<br />
I would like to express assignments in my DSL as <i>x := y</i> or <i>x <- y,</i> but I can't do that because C# won't let you create new infix operators. Hmm, well, I reckon << is hardly ever used, so maybe I could hijack <i>x << y</i> instead for assignment. It has as slightly tainted C++ flavour, but it's the best thing on offer.<!-----><!-----><!-----><br />
<br />
But, noooo, I can't do this because if I write<br />
<br />
<i>public static Act operator <<(Var</i><i><T></i><i><t>x, Expr</t></i><i><T></i><i><t><t>e) { ... }</t></t></i><br />
<i><br /></i>
I get a complaint from the C# compiler that <i>not only</i> is the first argument not of type <i>Expr</i><i><T></i> (the hosting class), but the second argument is not a plain old <i>int</i>! Great googly moogly, what were they thinking? Even if I wanted to implement <i><<</i> as bitwise left shift in my little DSL, I couldn't because of the idiotic second constraint. Instead I have to provide some awkward named function, <i>Set(x, e)</i> to do the job. Ugh.<br />
<br />
Damn you, C# !<br />
<br />
<b>Why would you expect any of this to work?</b><br />
<b><br /></b>
I just so happens I spent most of my life doing functional programming (I taught Haskell, I worked on the Mercury language team for a very long time). In sane, modern languages (i.e., ones invented since ~1970 by people who actually passed a few hard exams -- OO, I am not talking about you) this sort of thing is done as a matter of course.<br />
<br />
Arghghghghghgh.Rafehttp://www.blogger.com/profile/10169325688704651715noreply@blogger.com2tag:blogger.com,1999:blog-6767707.post-20000486485098317692013-08-13T19:57:00.001+09:302013-08-13T19:57:19.713+09:30Spolsky nails it<a href="http://www.joelonsoftware.com/articles/fog0000000319.html">http://www.joelonsoftware.com/articles/fog0000000319.html</a><br />
<br />
I have no idea what they teach in computing degrees other than a load of OO mumbo jumbo. It's depressing what industrial programmers haven't been taught.Rafehttp://www.blogger.com/profile/10169325688704651715noreply@blogger.com1tag:blogger.com,1999:blog-6767707.post-30240532497609369402012-02-04T08:18:00.000+10:302012-02-04T08:18:05.651+10:30LinqPadI've found something even better than the Snippet Compiler I blogged about recently: <a href="http://www.linqpad.net/">LinqPad</a>.<br />
<br />
LinqPad doesn't just act as a snippet compiler, it also acts as a marvellous database interface. Honestly, it's a joy to query a DB through Linq rather than SQL. LinqPad will also show you the SQL it generated for a given Linq query, which can be a real timesaver (SQL is a kitchen sink full of terrible syntactic special cases).Rafehttp://www.blogger.com/profile/10169325688704651715noreply@blogger.com1tag:blogger.com,1999:blog-6767707.post-92147256814016075372012-02-04T08:17:00.004+10:302012-02-04T08:17:54.602+10:30SQL in One Page (or thereabouts)I've written a very short summary of almost everything you need to know about SQL (one of the most dog-awful languages ever "designed"): <a href="https://docs.google.com/open?id=0BysHVTWyzoTXOWM0YTliMmEtZmNkNS00NzBiLWJmOTMtZWJkOTBkOTY5ZTNk">SQL in One Page (or thereabouts)</a>.Rafehttp://www.blogger.com/profile/10169325688704651715noreply@blogger.com0tag:blogger.com,1999:blog-6767707.post-27046818542155459402011-11-14T14:05:00.001+10:302011-11-14T14:07:22.149+10:30SnippetCompilerFrom time to time I need to try something out in C# in a few lines of code. It's always a bit of a pain setting up a new Visual Studio solution for the purpose (I've traditionally used a <i>PlayPen</i> project for this). I've recently discovered this little gem: <a href="http://www.sliver.com/dotnet/SnippetCompiler/">SnippetCompiler</a>. It's essentially a mini-IDE for just this situation. Smashing!Rafehttp://www.blogger.com/profile/10169325688704651715noreply@blogger.com0