FPGA synth

Heading towards a more reliable, deployable and software-leaning approach to working with video now.

The most plug and play option would appear to already exist : a tiny raspberry pi zero can already run Processing scripts and control small screens https://learn.adafruit.com/processing-on-the-raspberry-pi-and-pitft/processing !

We would be more in the zone of software video synth : https://learn.adafruit.com/feather-rp2040-dvi-video-synth

Also seem to be companies making just this kind of thing : https://store.excamera.com/

Arduino talks to the FPGA by SPI as if it were a RAM space. https://excamera.com/files/gameduino/synth/doc/gen/poster.pdf

Design / Engineer friend Paolo Salvagione pointed me to Configurable mixed-signal ICs. It appears to be like an FPGA but for mixed analog and digital circuitry and is configured in design software.

There is also the world of ASICs and the google open-source silicon project https://developers.google.com/silicon


My project was originally about slowing down computer processes and representing them in space. Now I just seem to order things on Amazon and design increasing numbers of PCBs. How could I get back to my original project here ?

My project also had a collection of related techniques :

  • Try to free machines from their corseted, factory settings and reveal their full range of expression that would otherwise just exist in a parallel unseen dimension
  • Try to show our files from the perspective of / through the eyes of the machines that are processing them.
  • Curate series of abstract formal compositions which emerge from figurative, architecturally-themed ones.
  • Try to describe the behaviour / idiosyncracies of algorithms and make it tangible
  • Isolate specific moments of machine-to-machine interface in a larger system
  • Get one’s hands inside the black box, and make prepared machines which expose their parameters
  • Try to rebuild electronic systems based on help from the DIY internet and then hope to stumble on something unintentional
  • Emphasize the materiality of technology
  • Trying to go super low-level and avoiding abstract computer “visualizations”
  • The link between visual chaos and harmony
  • Riffing off of the music synthesizer movement of knobs and playful electronics interfaces
  • Exploring the surreal, bizarre space of the computational unconscious
  • Do “research” projects that are educational

I also need to do a debrief on this leg of the video synth project. What have I learned ?

Technical stuff :

      • How to make and debug PCBs
      • To work with video + SRAM with counters as well as some op amp circuits. The difference between digital and analog in the video context.
      • To work with the VGA protocols, about resolution and brightness of the image.
      • I learned about oscillation, the difference between KHz and MHz in the context of video.

Stuff about media representation

      • synchronizing the data with the frame of the image is a key part of the illusion of video. If you mess with this suddenly the video frame becomes a malleable object. If it is desynchronized, the image is out of frame and will jump around with the SRAM recording. If you change the speed of recording or of playback you can zoom.
      • Layering of different images in a palimpsest can create textures from them.
      • I’ve made some nice abstract pattern discoveries through knob twisting.
      • Exploring the gap between legible figurative image and abstract pattern

I think the project started off about the screen itself, how to send signals and display them. At some point it became clear that the screen just displays data stored in memory.

In this spirit, I could try to represent memory access patterns : https://en.wikipedia.org/wiki/Memory_access_pattern Or the patterns that memory is refreshed in : https://en.wikipedia.org/wiki/Memory_refresh


I could also sample images and then modify them algorithmically? But it kind of ends up looking like simple Photoshop filters. It would have to be modified in some way based on the way the image is stored in the memory ?

Check out this image stored in DDR ram decaying over time from J. Alex Halderman’s paper : https://jhalderm.com/pub/papers/coldboot-sec08.pdf

Also it seems like different DDR has different decay rates (from A Trustworthy Key Generation Prototype Based on DDR3 PUF for Wireless Sensor Networks https://www.researchgate.net/figure/DRAM-cells-decay-feature-with-power-switch-interval-time-of-120-s-a-DDR3-device-1-b_fig3_263586301):



I’ve never thought of this before but I guess I could generate a simple arduino code, and then take the .hex file and glitch it with any of the glitching techniques I’ve been using and see what it puts on screen.

I could also test overclocking / overheating the atmel and seeing how they collapse !!



I should also finish the boards I’ve already designed and had made, they offer learning opportunities and possible discoveries ! The bucket brigade, the digitally-controlled pots, and the band splitter are the three that are left from that series. Also the Palimpsest.OS 2 board when controlled by microchip – especially if I can get the sync working ! Of course I have the new 128Kbit memory board once I get the parts too and then a final (?) FPGA v2 board.


I am still interested in exploring FPGAs and getting my hands in the tech to possibly stumble on cool stuff though so here is a second attempt at a video FPGA board based on the iCE40HX1K-TQ144 to output HDMI, DVI, VGA, and composite video. Goals for v2

For generating verilog, there is migen for python 3 (https://github.com/m-labs/migen). I’m going to try a super simple thing in Verilog first to get a feeling for it. 

I am learning verilog here : https://hdlbits.01xz.net/wiki/Main_Page

It looks like generating HDMI is doable even though there is a difference between LVDS and TMDS (involving possibly 100nF series capacitors ?) :




Different chips being used like the PTN3360D HDMI / DVI Level Shifter (IN STOCK!), a TFP410PAP TFP410 TI PanelBus™ Digital Transmitter (NO STOCK!),

Question - Pluto IIX GCvideo help needed | BitBuilt - Giving Life to Old Consoles

According to the datasheet, “The LVDS25E/subLVDSE differential output buffers are available on all banks but the LVDS/subLVDS input buffers are only available on Bank 3 ofiCE40 LP/HXdevices.”

I have replaced the LDO with was giving me trouble previously and am making this version USB programmable so it could be a useful for others as a cool video kit.


Revisiting FPGA programming, here is where I got to last time :

Here is my process for the FPGA programming so far :

  1. I signed up for an account at www.latticesemi.com.
  2. I got a licence for iCEcube2, put it somewhere easy to find and took note of the directory, and downloaded iCEcube2 and the Lattice Diamond software from here : https://www.latticesemi.com/en/Products/DesignSoftwareAndIP
  3. I plugged in the iCEstick and it blinked in a circular pattern. I also checked that it had a port under Device Manager (it had 2!). 
  4. I went into iCEcube2 then to Help > Tutorial to open a pdf called “Implementing the Design” and followed the following steps :
    1. Start new project (iCE40, HX1K, ?TQ144?)
    2. Add Design file iCElab.v and Constraint file iCElab.sdc and checked the Device Info.
    3. Run Synplify Pro Synthesis
    4. Select Implementation
    5. Add constraint file iCElab.pcf under Import P&R Input Files.
    6. Run Placer
    7. Check the Floorplan, package view, timing analysis, power estimation
    8. Generate the bitmap (.bit and .hex files) which is saved wherever the project was saved. (For instance C:\Users\Jonah\Desktop\FPGA LATTICE DEMO PROJECTS\quick_start\quick_start\quick_start_Implmnt\sbt\outputs\bitmap).
    9. Here is what everything should look like once finished :
  5. I then transitioned to Youtube for a detailed explaination of using Diamond Programmer with the iCEstick here : https://www.youtube.com/watch?v=Df9k1T0bHmA&ab_channel=Dom
    1. Launch Diamond Programmer and take the default options (or click scan and take those). Important to note that you must launch Lattice Diamond Programmer directly, not through Lattice Diamond, in order to find the iCE40HX1K chip.
    2. It will mistake the board for a JTAG interface and give the following errors : “Failed to scan board”, “Scan Failed – Creating Blank Programmer Project.”
    3. Now change the Device Family to iCE40, the device to iCE40HX1K, and under Program enter the following information :
      1. Change Access mode to SPI Flash Programming and now some other options appear. Select > Vendor: Micron, Device: N25Q032, Package:  8-pin VDFPN8
      2. Now load the bitmap (.bin) file and check that it has a non-zero datasize :
      3. Click OK and then click the green traffic light on the next to top row of icons. You should then see a completion message and have the program on your board :
      4. Presto !

Just for info :

For the actual pinout of the ice40 (which you can change in Pin Constraints Editor):


And here is my attempt programming my own board with the Lattice Programmer.

I followed the same steps as above but instead selected the HW-USBN-2B (FTDI) programmer. Important to note that you must launch Lattice Diamond Programmer directly, not through Lattice Diamond, in order to find the iCE40HX1K chip. Also note that you need Lattice iCEcube2™ Software, not just Diamond for some reason.

Here are the color connections needed from the Lattice Programming Cables User Guide :

The USB power and Board power lights should be on on the programmer and the Prog blue LED should flash a few times before the Done blue LED will flash and Lattice Programmer will display Operation: successful.

I am taking the example LED rotation verilog .v code :

module LED_Rotation(

    input  clk,
    output LED1,
    output LED2,
    output LED3,
    output LED4,
    output LED5


               reg[15:0] div_cntr1;
               reg[6:0] div_cntr2;
               reg[1:0] dec_cntr;
               reg half_sec_pulse;                         

               always@(posedge clk)
                              div_cntr1 <= div_cntr1 + 1;
                              if (div_cntr1 == 0)
                                            if (div_cntr2 == 91)

                                                           div_cntr2 <= 0;
                                                           half_sec_pulse <= 1; 
                                                           div_cntr2 <= div_cntr2 + 1;
                                            half_sec_pulse <= 0;                            

                              if (half_sec_pulse == 1) 
                                            dec_cntr <= dec_cntr + 1;                                          
               assign LED1 = (dec_cntr == 0) ;
               assign LED2 = (dec_cntr == 1) ;
               assign LED3 = (dec_cntr == 2) ;
               assign LED4 = (dec_cntr == 3) ;
               assign LED5 = 1'b1;

Under Tool > Pin Constraints Editor, we can change the pins activated in the code but mine is already set up connected to pin 99:

I am soldering a second FPGA board. I noticed that the clock was not outputing anything, and also that the max temperature for the IC for reflow soldering is around 250°C. I am making sure to use the soldering iron “kissing” technique along with loads of flux applied.

Reading through the datasheet :

  • Bank 3 additionally supports differential LVDS25 input buffers.
  • Each bank can have it’s own voltage level.
  • I may have to power the VCC, VCCIO_2, VCC_SPI, VCC_PLL, and VPP_2V5 (with a diode) even if I don’t plan on using them all just to get the device to turn on. (There is even a proper sequence to power them up in).
  • GNDPLL must NOT be connected to the board’s ground !
  • Bank 2 has two dedicated Input pins.



  • 1.2V is connected to 5V throughout because of an EAGLE issue 🙁
  • I tried the voltage regulator 3 times and still no luck. *UPDATE* I tried a second 1.2V LDO that I ordered and it still reads 2.2V or so. No idea why this is happening when I can power the pin directly with 1.2V with my power supply no problem (therefore it can’t be a bad connection somewhere between 1.2V and another trace right?). Positive news is that the FPGA appears to have survived the over-voltage !!
  • Maybe I should give it a higher frequency oscillator, at least 25MHz, as I’m working with video?
  • I connected pin 9 of VGA to GND, it is connected to 5V on the signal side !
  • The GNDPLL should not be connected to GND.
  • The R2R ladder has a 270ohm going to ground instead of a 536 on each ladder.
  • Unclear which pins to plug where for programming – *EDIT* SOLVED
  • I had a hard time at first soldering the FPGA but then discovered a soldering “kiss” technique that was reliable and effective. For the clock, I melted the solder and then placed it gently on top.
  • I plugged it in with 2.2V instead of 1.2V and likely destroyed the chip the first time…
  • 0805 is doable and better for compact designs
  • I can’t find the name of the chip in the Target Device menu of Diamond Programmer. *EDIT* SOLVED : You need to open Diamond Programmer directly.
  • I need to get nicer boards if I am going to be working on FPGA boards, Aisler not chinese manufactured.
  • It seems like exploiting the parallelism of the FPGA could be interesting to explore. For this project it would probably be to logically mix several input videos together while transforming ?


I’ve got it blinking at different frequencies !

Now I want to generate some VGA signals as quickly as possible and start creating images by tweaking pre-made code like for VGAX.

And here is demo code for the OLIMEX ice40 developement board which has RAM, DAC, ADC and VGA (but it needs a 100MHz clock) : https://github.com/OLIMEX/iCE40HX1K-EVB/tree/master/demo/ice40-io-video

Also super simple (but uses a 25MHz clock) : https://www.fpga4fun.com/PongGame.html

Here is the one I ended up using (takes a 12MHz clock and turns it into a 25MHz with the PLL) : https://github.com/imuguruza/alhambra_II_test/blob/master/vga/vga_test/

Here is the key part of the code that actually draws on screen :

module vga_sync_test(
    input wire clk_in,
    input wire reset,
    output reg r0,
    output reg r1,
    output reg r2,
    output reg b0,
    output reg b1,
    output reg b2,
    output reg g0,
    output reg g1,
    output reg g2,
    output wire h_sync,
    output wire v_sync,
    output wire led,
    output wire locked_led

wire clk_sys;
wire display_en;
//reg [9:0] h_count;
wire [9:0] h_count;
//reg [9:0] v_count;
wire [9:0] v_count;
assign  led = clk_sys;

localparam  h_pixel_max = 640;
localparam  v_pixel_max = 480;
localparam  h_pixel_half = 320;
localparam  v_pixel_half = 240;

//Check if we can create RGB colors
always @(posedge clk_sys) begin
  if (display_en) begin
    if (h_count < h_pixel_half
        && v_count < v_pixel_half) begin
      //Assign here your test color
      r0 <= 1'b0;
      r1 <= 1'b0;
      r2 <= 1'b0;
      g0 <= 1'b0;
      g1 <= 1'b0;
      g2 <= 1'b0;
      b0 <= 1'b1;
      b1 <= 1'b1;
      b2 <= 1'b1;
    end else if (h_count > h_pixel_half
            && v_count < v_pixel_half) begin
      //Assign here your test color
      r0 <= 1'b0;
      r1 <= 1'b0;
      r2 <= 1'b0;
      g0 <= 1'b1;
      g1 <= 1'b1;
      g2 <= 1'b1;
      b0 <= 1'b0;
      b1 <= 1'b0;
      b2 <= 1'b0;
     end else if (h_count < h_pixel_half
            && v_count > v_pixel_half) begin
      //Assign here your test color
      r0 <= 1'b1;
      r1 <= 1'b1;
      r2 <= 1'b1;
      g0 <= 1'b0;
      g1 <= 1'b0;
      g2 <= 1'b0;
      b0 <= 1'b0;
      b1 <= 1'b0;
      b2 <= 1'b0;
    end else begin
      //Assign here your test color
      r0 <= 1'b1;
      r1 <= 1'b1;
      r2 <= 1'b1;
      g0 <= 1'b1;
      g1 <= 1'b1;
      g2 <= 1'b1;
      b0 <= 1'b1;
      b1 <= 1'b1;
      b2 <= 1'b1;
  end else begin
    r0 <= 1'b0;
    r1 <= 1'b0;
    r2 <= 1'b0;
    g0 <= 1'b0;
    g1 <= 1'b0;
    g2 <= 1'b0;
    b0 <= 1'b0;
    b1 <= 1'b0;
    b2 <= 1'b0;

vga_sync vga_s(
      .clk_in(clk_in),         //12MHz clock input
      .reset(reset),           // RST assigned to SW1
      .clk_sys(clk_sys),       //25.125 MHz clock generated by PLL
      .display_en(display_en), // '1' => pixel region
      .locked(locked_led)      // PLL signal, '1' => OK


I would like to modify this code in the spirit of VGAX bytebeat-inspired video synthesis.

I made sure to load the three dependent .v files and then changed the pin constraints in the .pcf as follows :

set_io b0 76
set_io clk_in 21
set_io led 99
set_io r0 81
set_io b1 75
set_io g2 91
set_io r1 80
set_io v_sync 87
set_io b2 74
set_io g1 95
set_io locked_led 142
set_io r2 79
set_io reset 2
set_io g0 96
set_io h_sync 88


It works !

Just waiting for the LDOs to make it more practical to plug in. I will now test some actual patterns !


Here is my first test bench with ModelSim ! Thanks to this tutorial from Nandland : https://nandland.com/tutorial-your-fpga-program-an-led-blinker-part-2/

I am using ModelSim Lattice FPGA 2020.3 (which downloaded with either Icecube2 or Lattice Programmer?) and followed this PDF called
Using Mentor ModelSim Simulator with Lattice iCEcube2

The steps to follow :

  1. Open a new project
  2. Click Add Existing Files (pick only the test bench verilog file) and then Close
  3. Now Compilation > Compile All
  4. Now Simulation > Start Simulation
  5. In the console type <<view wave>>…
  6. …then <<add wave * >>…
  7. …and a time period for example : <<run 100ms>>. (It takes us, ns, ms, and sec it appears)

The signals are a little small and it is apparently possible to change this under Tools -> Edit Preferences, Wave Font and Footer Font…but when I try it crashes !

Looks like printing to the console is going to be useful ! They work by using display like so…

$display("0.5 seconds");

…and then in the console area of ModelSim:

run -all

Here’s a simpler version of a tb for a 10 ps period of the following module :

module dut ( input clk ); 
`timescale 1ps / 1ps
module top_module ( output reg clk );

      dut instance2 (.clk(clk));

           initial begin
           clk = 1'b0;
           forever #5 clk = ~clk;


Thinking about different ways of generating patterns on a screen :

  1. Screen coordinate based (plotting functions that take an input x and produce an output y)
  2. Time based + screen coordinate based (a counter t increases over time and modifies the function’s behaviour)

Turns out FPGAs can’t do math like microprocessors ! To implement trig functions you need to build a CORDIC, which is like a computer to calculate functions.

Then again there are some things FPGAs can do better than microcontrollers like cellular automata (one study I checked out said 36 times faster than a GPU and up to 2,800 times faster than software). HDL Bits has an exercise to build the rule 30, 90, rule 101, rule 184 cellular automata.

Here was the solution to the rule 90 challenge that I had to look up in the end :

module top_module(
input clk,
input load,
input [511:0] data,
output reg [511:0] q);

always @(posedge clk) begin
       if (load)
           q <= data; // Load the DFFs with a value.
       else begin
          // At each clock, the DFF storing each bit position becomes the XOR of its left neighbour
          // and its right neighbour. Since the operation is the same for every
          // bit position, it can be written as a single operation on vectors.
          // The shifts are accomplished using part select and concatenation operators.

         // left                     right
         // neighbour              neighbour
           q <= q[511:1] ^ {q[510:0], 1'b0} ;

Check out this processing sketch that is evocative of cellular automata :

https://openprocessing.org/sketch/1879926 or sketch 1908904

Could it be possible to do this live to recorded video with the FPGA and be able to vary the number of generations, speed, etc. ?

I am now trying to use the test bench to better understand why this code isn’t putting any pixels on the screen currently.

Here is my test bench. An important note is that variables that are not modified in an always or initial block should not be reg but wire instead.

`include "rule90.v"
`timescale 1ps / 1ps

module rule90_tb();
reg clk;
reg load;
reg [511:0] data;
wire [511:0] q;

rule90 DUT (

initial begin
clk = 1'b1;
clk = 1'b0;
load = 1'b1;
data = 1'b1;
clk = 1'b1;
clk = 1'b0;
load = 1'b0;


always #10 clk = ~clk;

This shows that the rule90 circuit is working. Every clock cycle it advances. If I could show one bit per pixel and refresh every clock cycle ?


For the next FPGA board, I’m aiming for the following features :

  • FTDI chip for USB programming
  • 100MHz oscillator
  • SRAM
  • HDMI out, possibly composite and DVI also ?
  • Some kind of basic interface ? (Knobs etc?)
  • An atmega which can ask the FPGA to do things?
  • A fan and heat sink for the FPGA??
  • A fuse and reverse polarity protection !
  • microcontroller-accessed SPI FLASH so we can test moving between different types of memory

Reading a bit more about what FPGAs can do well ;

  • low latency applications that take data as a stream in real time from sensors, cameras, etc.
  • high throughput applications, pipelining
  • encryption and networking
  • digital signal processing (IIR, FIR, FFT, ADC, DAC) like video filtering
  • parallelism, like doing 128 small tasks at the same time, versus microcontrollers doing things in serial
  • algorithms that can be solved with LUTs, tasks with lots of small adding and subtracting or multiplication (?)
  • K-means clustering, Monte Carlo, neural nets, Mandelbrot fractals, cellular automata ?
  • Seems that there are more and more systems that include micrcontrollers + FPGA + DSP, and you can of course put a microchip inside an FPGA. Here the FPGA accelerates certain tasks for the CPU.