FPGA synth

Heading towards a more reliable, deployable and software-leaning approach to working with video now.

The most plug and play option would appear to already exist : a tiny raspberry pi zero can already run Processing scripts and control small screens https://learn.adafruit.com/processing-on-the-raspberry-pi-and-pitft/processing !

We would be more in the zone of software video synth : https://learn.adafruit.com/feather-rp2040-dvi-video-synth

Also seem to be companies making just this kind of thing : https://store.excamera.com/

Arduino talks to the FPGA by SPI as if it were a RAM space. https://excamera.com/files/gameduino/synth/doc/gen/poster.pdf

Design / Engineer friend Paolo Salvagione pointed me to Configurable mixed-signal ICs. It appears to be like an FPGA but for mixed analog and digital circuitry and is configured in design software.

There is also the world of ASICs and the google open-source silicon project https://developers.google.com/silicon

*************

My project was originally about slowing down computer processes and representing them in space. Now I just seem to order things on Amazon and design increasing numbers of PCBs. How could I get back to my original project here ?

My project also had a collection of related techniques :

• Try to free machines from their corseted, factory settings and reveal their full range of expression that would otherwise just exist in a parallel unseen dimension
• Try to show our files from the perspective of / through the eyes of the machines that are processing them.
• Curate series of abstract formal compositions which emerge from figurative, architecturally-themed ones.
• Try to describe the behaviour / idiosyncracies of algorithms and make it tangible
• Isolate specific moments of machine-to-machine interface in a larger system
• Get one’s hands inside the black box, and make prepared machines which expose their parameters
• Try to rebuild electronic systems based on help from the DIY internet and then hope to stumble on something unintentional
• Emphasize the materiality of technology
• Trying to go super low-level and avoiding abstract computer “visualizations”
• The link between visual chaos and harmony
• Riffing off of the music synthesizer movement of knobs and playful electronics interfaces
• Exploring the surreal, bizarre space of the computational unconscious
• Do “research” projects that are educational

I also need to do a debrief on this leg of the video synth project. What have I learned ?

Technical stuff :

• • • How to make and debug PCBs
    • To work with video + SRAM with counters as well as some op amp circuits. The difference between digital and analog in the video context.
    • To work with the VGA protocols, about resolution and brightness of the image.
    • I learned about oscillation, the difference between KHz and MHz in the context of video.

Stuff about media representation

• • • synchronizing the data with the frame of the image is a key part of the illusion of video. If you mess with this suddenly the video frame becomes a malleable object. If it is desynchronized, the image is out of frame and will jump around with the SRAM recording. If you change the speed of recording or of playback you can zoom.
    • Layering of different images in a palimpsest can create textures from them.
    • I’ve made some nice abstract pattern discoveries through knob twisting.
    • Exploring the gap between legible figurative image and abstract pattern

I think the project started off about the screen itself, how to send signals and display them. At some point it became clear that the screen just displays data stored in memory.

In this spirit, I could try to represent memory access patterns : https://en.wikipedia.org/wiki/Memory_access_pattern Or the patterns that memory is refreshed in : https://en.wikipedia.org/wiki/Memory_refresh

http://www.overbyte.com.au/misc/Lesson3/CacheFun.html

I could also sample images and then modify them algorithmically? But it kind of ends up looking like simple Photoshop filters. It would have to be modified in some way based on the way the image is stored in the memory ?

Check out this image stored in DDR ram decaying over time from J. Alex Halderman’s paper : https://jhalderm.com/pub/papers/coldboot-sec08.pdf

Also it seems like different DDR has different decay rates (from A Trustworthy Key Generation Prototype Based on DDR3 PUF for Wireless Sensor Networks https://www.researchgate.net/figure/DRAM-cells-decay-feature-with-power-switch-interval-time-of-120-s-a-DDR3-device-1-b_fig3_263586301):

********

I’ve never thought of this before but I guess I could generate a simple arduino code, and then take the .hex file and glitch it with any of the glitching techniques I’ve been using and see what it puts on screen.

I could also test overclocking / overheating the atmel and seeing how they collapse !!

********

I should also finish the boards I’ve already designed and had made, they offer learning opportunities and possible discoveries ! The bucket brigade, the digitally-controlled pots, and the band splitter are the three that are left from that series. Also the Palimpsest.OS 2 board when controlled by microchip – especially if I can get the sync working ! Of course I have the new 128Kbit memory board once I get the parts too and then a final (?) FPGA v2 board.

********

I am still interested in exploring FPGAs and getting my hands in the tech to possibly stumble on cool stuff though so here is a second attempt at a video FPGA board based on the iCE40HX1K-TQ144 to output HDMI, DVI, VGA, and composite video. Goals for v2

For generating verilog, there is migen for python 3 (https://github.com/m-labs/migen). I’m going to try a super simple thing in Verilog first to get a feeling for it. 

I am learning verilog here : https://hdlbits.01xz.net/wiki/Main_Page

It looks like generating HDMI is doable even though there is a difference between LVDS and TMDS (involving possibly 100nF series capacitors ?) :

https://www.fpga4fun.com/HDMI.html

blackmesalabs.wordpress.com/2017/12/15/bml-hdmi-video-for-fpgas-over-pmod/

https://github.com/hdl-util/hdmi/

Different chips being used like the PTN3360D HDMI / DVI Level Shifter (IN STOCK!), a TFP410PAP TFP410 TI PanelBus™ Digital Transmitter (NO STOCK!),

According to the datasheet, “The LVDS25E/subLVDSE differential output buffers are available on all banks but the LVDS/subLVDS input buffers are only available on Bank 3 ofiCE40 LP/HXdevices.”

I have replaced the LDO with was giving me trouble previously and am making this version USB programmable so it could be a useful for others as a cool video kit.

*****

Revisiting FPGA programming, here is where I got to last time :

Here is my process for the FPGA programming so far :

1. I signed up for an account at www.latticesemi.com.
2. I got a licence for iCEcube2, put it somewhere easy to find and took note of the directory, and downloaded iCEcube2 and the Lattice Diamond software from here : https://www.latticesemi.com/en/Products/DesignSoftwareAndIP
3. I plugged in the iCEstick and it blinked in a circular pattern. I also checked that it had a port under Device Manager (it had 2!). 
4. I went into iCEcube2 then to Help > Tutorial to open a pdf called “Implementing the Design” and followed the following steps :
   1. Start new project (iCE40, HX1K, ?TQ144?)
   2. Add Design file iCElab.v and Constraint file iCElab.sdc and checked the Device Info.
   3. Run Synplify Pro Synthesis
   4. Select Implementation
   5. Add constraint file iCElab.pcf under Import P&R Input Files.
   6. Run Placer
   7. Check the Floorplan, package view, timing analysis, power estimation
   8. Generate the bitmap (.bit and .hex files) which is saved wherever the project was saved. (For instance C:\Users\Jonah\Desktop\FPGA LATTICE DEMO PROJECTS\quick_start\quick_start\quick_start_Implmnt\sbt\outputs\bitmap).
   9. Here is what everything should look like once finished :
   10. 
5. I then transitioned to Youtube for a detailed explaination of using Diamond Programmer with the iCEstick here : https://www.youtube.com/watch?v=Df9k1T0bHmA&ab_channel=Dom
   1. Launch Diamond Programmer and take the default options (or click scan and take those). Important to note that you must launch Lattice Diamond Programmer directly, not through Lattice Diamond, in order to find the iCE40HX1K chip.
   2. It will mistake the board for a JTAG interface and give the following errors : “Failed to scan board”, “Scan Failed – Creating Blank Programmer Project.”
   3. Now change the Device Family to iCE40, the device to iCE40HX1K, and under Program enter the following information :
      1. Change Access mode to SPI Flash Programming and now some other options appear. Select > Vendor: Micron, Device: N25Q032, Package:  8-pin VDFPN8
      2. Now load the bitmap (.bin) file and check that it has a non-zero datasize :
      3. 
      4. Click OK and then click the green traffic light on the next to top row of icons. You should then see a completion message and have the program on your board :
      5. 
      6. Presto !

Just for info :

For the actual pinout of the ice40 (which you can change in Pin Constraints Editor):

***

And here is my attempt programming my own board with the Lattice Programmer.

I followed the same steps as above but instead selected the HW-USBN-2B (FTDI) programmer. Important to note that you must launch Lattice Diamond Programmer directly, not through Lattice Diamond, in order to find the iCE40HX1K chip. Also note that you need Lattice iCEcube2™ Software, not just Diamond for some reason.

Here are the color connections needed from the Lattice Programming Cables User Guide :

The USB power and Board power lights should be on on the programmer and the Prog blue LED should flash a few times before the Done blue LED will flash and Lattice Programmer will display Operation: successful.

I am taking the example LED rotation verilog .v code :

module LED_Rotation(

    input  clk,
    output LED1,
    output LED2,
    output LED3,
    output LED4,
    output LED5

    );

               reg[15:0] div_cntr1;
               reg[6:0] div_cntr2;
               reg[1:0] dec_cntr;
               reg half_sec_pulse;                         

               always@(posedge clk)
                              begin
                              div_cntr1 <= div_cntr1 + 1;
                              if (div_cntr1 == 0)
                                            if (div_cntr2 == 91)

                                                           begin
                                                           div_cntr2 <= 0;
                                                           half_sec_pulse <= 1; 
                                                           end
                                            else
                                                           div_cntr2 <= div_cntr2 + 1;
                              else
                                            half_sec_pulse <= 0;                            

                              if (half_sec_pulse == 1) 
                                            dec_cntr <= dec_cntr + 1;                                          
                              end                                                            
               assign LED1 = (dec_cntr == 0) ;
               assign LED2 = (dec_cntr == 1) ;
               assign LED3 = (dec_cntr == 2) ;
               assign LED4 = (dec_cntr == 3) ;
               assign LED5 = 1'b1;
                                                     
endmodule

module vga_sync_test(
    input wire clk_in,
    input wire reset,
    output reg r0,
    output reg r1,
    output reg r2,
    output reg b0,
    output reg b1,
    output reg b2,
    output reg g0,
    output reg g1,
    output reg g2,
    output wire h_sync,
    output wire v_sync,
    output wire led,
    output wire locked_led
  );

wire clk_sys;
wire display_en;
//reg [9:0] h_count;
wire [9:0] h_count;
//reg [9:0] v_count;
wire [9:0] v_count;
assign  led = clk_sys;

localparam  h_pixel_max = 640;
localparam  v_pixel_max = 480;
localparam  h_pixel_half = 320;
localparam  v_pixel_half = 240;

//Check if we can create RGB colors
always @(posedge clk_sys) begin
  if (display_en) begin
    if (h_count < h_pixel_half
        && v_count < v_pixel_half) begin
      //Assign here your test color
      r0 <= 1'b0;
      r1 <= 1'b0;
      r2 <= 1'b0;
      g0 <= 1'b0;
      g1 <= 1'b0;
      g2 <= 1'b0;
      b0 <= 1'b1;
      b1 <= 1'b1;
      b2 <= 1'b1;
    end else if (h_count > h_pixel_half
            && v_count < v_pixel_half) begin
      //Assign here your test color
      r0 <= 1'b0;
      r1 <= 1'b0;
      r2 <= 1'b0;
      g0 <= 1'b1;
      g1 <= 1'b1;
      g2 <= 1'b1;
      b0 <= 1'b0;
      b1 <= 1'b0;
      b2 <= 1'b0;
     end else if (h_count < h_pixel_half
            && v_count > v_pixel_half) begin
      //Assign here your test color
      r0 <= 1'b1;
      r1 <= 1'b1;
      r2 <= 1'b1;
      g0 <= 1'b0;
      g1 <= 1'b0;
      g2 <= 1'b0;
      b0 <= 1'b0;
      b1 <= 1'b0;
      b2 <= 1'b0;
    end else begin
      //Assign here your test color
      r0 <= 1'b1;
      r1 <= 1'b1;
      r2 <= 1'b1;
      g0 <= 1'b1;
      g1 <= 1'b1;
      g2 <= 1'b1;
      b0 <= 1'b1;
      b1 <= 1'b1;
      b2 <= 1'b1;
      end
  end else begin
    r0 <= 1'b0;
    r1 <= 1'b0;
    r2 <= 1'b0;
    g0 <= 1'b0;
    g1 <= 1'b0;
    g2 <= 1'b0;
    b0 <= 1'b0;
    b1 <= 1'b0;
    b2 <= 1'b0;
  end
end

vga_sync vga_s(
      .clk_in(clk_in),         //12MHz clock input
      .reset(reset),           // RST assigned to SW1
      .h_sync(h_sync),
      .v_sync(v_sync),
      .clk_sys(clk_sys),       //25.125 MHz clock generated by PLL
      .h_count(h_count),
      .v_count(v_count),
      .display_en(display_en), // '1' => pixel region
      .locked(locked_led)      // PLL signal, '1' => OK
      );

endmodule

set_io b0 76
set_io clk_in 21
set_io led 99
set_io r0 81
set_io b1 75
set_io g2 91
set_io r1 80
set_io v_sync 87
set_io b2 74
set_io g1 95
set_io locked_led 142
set_io r2 79
set_io reset 2
set_io g0 96
set_io h_sync 88

$display("0.5 seconds");

run -all

module dut ( input clk ); 

`timescale 1ps / 1ps
module top_module ( output reg clk );

      dut instance2 (.clk(clk));

           initial begin
           clk = 1'b0;
           forever #5 clk = ~clk;
     end
endmodule

module top_module(
input clk,
input load,
input [511:0] data,
output reg [511:0] q);

always @(posedge clk) begin
       if (load)
           q <= data; // Load the DFFs with a value.
       else begin
          // At each clock, the DFF storing each bit position becomes the XOR of its left neighbour
          // and its right neighbour. Since the operation is the same for every
          // bit position, it can be written as a single operation on vectors.
          // The shifts are accomplished using part select and concatenation operators.

         // left                     right
         // neighbour              neighbour
           q <= q[511:1] ^ {q[510:0], 1'b0} ;
        end
    end
endmodule

`include "rule90.v"
`timescale 1ps / 1ps


module rule90_tb();
reg clk;
reg load;
reg [511:0] data;
wire [511:0] q;

rule90 DUT (
.clk(clk),
.load(load),
.data(data),
.q(q)
);

initial begin
#10
clk = 1'b1;
#10;
clk = 1'b0;
#10;
load = 1'b1;
data = 1'b1;
clk = 1'b1;
#10;
clk = 1'b0;
#10;
load = 1'b0;

end

always #10 clk = ~clk;
endmodule