KiCad SKiDL best practices

[obiwanjacobi]

I am playing around with SKiDL and run into some 'issues'. I was wondering if there is anybody out there that also does programmatic net list generation and has figured out these issues.
Must be said that I am new to Python so there may be some obvious solutions that I did not think of.

• How do you structure your code in order to keep it readable?
  I know you can make functions and for loops etc. but most code turns out to be a sequential list of connecting parts...
• How do you manage part naming (annotation)?
  I have adopted having a Bom file that uses arrays for each type to pre-create all parts. This will name them in order. Then connect them with something like R[2][1] += D[2].A - this would connect R2 with the Anode (alias) of D2.
• How do you validate correctness?
  After you have coded your net list, how do you know for sure that its all good - besides running rules. Simply start laying out the PCB and run into the errors there? (yak!)

I am working from an existing schematic (Harlequin one of the ZX Spectrum clones) which is fairly large - not trivial in any case.

Thoughts?

Edit: code examples.

Code: [Select]

# libraries
device = 'device'

# footprints
fpR = 'Resistor_SMD:R_0805_2012Metric'
fpC = 'Capacitor_SMD:C_0805_2012Metric'
fpD = 'Diode_SMD:R_0805_2012Metric'

R = [{}]
C = [{}]
D = [{}]
# zero indexes are never used
for r in range(65):
    R.append(Part(device, 'R' , footprint=fpR))
for c in range(45):
    C.append(Part(device, 'C', footprint=fpC))
for d in range(10):
    D.append(Part(device, 'D', footprint=fpD))

C[1].value = '10uF'
C[3].value = '10uF'
C[27].value = '10uF'
C[2].value = '27pF'
C[4].value = '27pF'
C[25].value = '10pF'
...


Code: [Select]

# global nets
vcc = Net('VCC')
gnd = Net('GND')

busD = Bus('D', 8)
busA = Bus('A', 16)

# z80 signals
clk = Net('CLK')
rst = Net('RST')
rd = Net('RD')
wr = Net('WR')
mreq = Net('MREQ')
iorq = Net('IORQ')
rfsh = Net('RFSH')
m1 = Net('M1')
nmi = Net('NMI')
int = Net('INT')
busrq = Net('BUSRQ')
busack = Net('BUSACK')
wait = Net('WAIT')
halt = Net('HALT')

# CPU and memory

z80 = U[1]
z80['VCC'] += vcc
z80['GND'] += gnd
z80['D[0:7]'] += busD
z80['A[0:15]'] += busA
z80['~RESET'] += rst
z80['~CLK'] += clk
z80['~IORQ'] += iorq
z80['~RD'] += rd
z80['~WR'] += wr
z80['~BUSRQ'] += busrq
z80['~BUSACK'] += busack
z80['~WAIT'] += wait
z80['~INT'] += int
z80['~NMI'] += nmi
z80['~M1'] += m1
z80['~HALT'] += halt
z80['~RFSH'] += rfsh
z80['~MREQ'] += mreq

rom = U[2]
rom['VCC'] += vcc
rom['GND'] += gnd
rom['D[0:7]'] += busD
rom['A[0:13]'] += busA['A[0:13]']
#rom['A[14:15'] += busA
#rom['~WE'] += vcc
rom['~OE'] += rd
rom['~CE'] += Net.fetch('ROMCS')

ram = U[3]
ram['VCC'] += vcc
ram['GND'] += gnd
ram['Q[0:7]'] += busD
ram['A[0:13]'] += busA['A[0:13]']
ram['A[14]'] += Net.fetch('RA14')
ram['A[15]'] += Net.fetch('RA15')
ram['~OE'] += rd
ram['~WE'] += wr
ram['~CS1'] += Net.fetch('RAM')
ram['CS2'] += busA[15]


[SiliconWizard]

This sort of follows up on this discussion: https://www.eevblog.com/forum/eda/no-schematic-pcb-workflow/

After that, I actually got to design my own tool (I found Skidl interesting but 1/ didn't like Python much and 2/ thought there were things lacking for it to be really usable, at least by me). My tool is based on Lua and I codenamed it "Lucid" (loosely for Lua-based Circuit Design) for the time being. I'm not open-sourcing it for now (might change later on), so I'm sorry I can't share it yet. I designed a board with it (an isolated SWD/JTAG probe with USB-C) and used KiCad to route it. It turned out pretty good. Just a quick picture attached (yes there is a manual correction as can be seen, but it was due to some detail I overlooked while designing it, and has actually nothing to do with my tool itself, I would have made the same mistake with a schematic).

So anyway, I can give you a couple hints since I have this experience, but I have none with Skidl so I can't comment on what can or can't be done with it.

1. The key IMO with a schematic-less design is (as we also discussed in the above thread) to use hierarchy as much as you can, so you can deal with sub-blocks that you can then instantiate and connect in higher-level blocks. That keeps the design tidy and readable, and keeps the number of parts in each sub-block manageable. You can of course also use any feature of the underlying language (Python in your case), but the key really is to segment your design in small blocks. If some blocks are similar but with small variations (component value or otherwise), you can of course write functions to generate those blocks with parameters. Pretty handy. Also, if you can, when dealing with identical parts (like a series of 100nF decoupling caps, make clones of one rather than create each one (which is very annoying and error-prone). Such as in this example you gave:

Code: [Select]

C[1].value = '10uF'
C[3].value = '10uF'
C[27].value = '10uF'
I'd suggest creating a 10uF capacitor first, and then making copies for all the identical ones (it can probably be done with Skidl/Python).

2. Not sure about how Skidl works here, but annotation was certainly a big concern of mine (as again discussed in the previous thread). So with my tool, annotation is automatic. What it means is that you name the parts in each block as you see fit (with identifiers that make sense to you, a bit like naming variables), and while analyzing the circuit, my tool will automatically generate conventional references (such as R1, C2, U5...) from the internal, hierarchical part/block names, and generate a "dictionary" file (internal names vs. annotated references in text files). I find it actually pretty usable. Finding a reference later on is dead easy (especially since the dictionaries are sorted), and since the internal names are hierarchical, it's easy to spot where a given reference lies in your circuit. I'm not sure Skidl can do that, and that would be the most annoying thing IMO.

3. Good question, but how do you validate correctness with a schematic? The key here again is to make your design "readable", since it's much harder to figure out a circuit from a textual representation rather than from a graphical one. So keep it very hierarchical and validate it by relatively small sub-blocks. I don't know whether Skidl has ERC (I think so), I added that in my own tool and it's invaluable: you can definitely not do without ERC when going schematic-less. Much too easy to forget connections for instance, so ERC should definitely give you the unconnected ports. To ease verification (and communication with others), I've started working on automatic generation of some kind of block diagrams. Not that easy. Something like the rtl-to-schematic tools you can find in many FPGA IDEs would be handy, but it's definitely not so simple to write and get something visually useful. So I'm not there yet.

[obiwanjacobi]

Thanks for your response.

1+2) Because I am following an existing schematic, I wanted to keep the annotation in sync. So R1 in my 'program' had to be the same R1 in the schematic. You either have to set the annotation individually, or its value (at least have not found another way). So I opted to pre-create a whole bunch of R's into an array up front, which will be named in order. So R[1] is 'R1', R[2] is 'R2' etc. That is why I had to assign values separately. I guess if you did not have this restriction, you could use 'factories' to create similar components.
For the IC's this trick does not work, so I have a whole list of creating specific 74-series logic chips in the Bom file.

On hierarchies: I thought about that, but I did not decide on how to implement it in detail. My concern is the number of interconnects between the hierarchical blocks (sub-circuits). I also found out later that SKiDL does have a Circuit class could perhaps be used to create these sub-circuits and add them together at a higher level. The schematic does not have any repeating blocks which could be implemented with a factory method, as you suggested.

3) Yes, SKiDL does execute the rules (I think the same as what KiCad uses), but I could imagine you can make a whole class of different errors in code, these rules are not prepared for? I was thinking in terms of unit-tests for my sub-circuits..?
As I only have done a small portion of the complete schematic I am working from, I could only imagine the chance of me doing it wrong (undetected) would increase significantly...

[SiliconWizard]

If you're going to translate an existing schematic and want to keep the references, then yes just use the same references, but again I can't comment on how to do that properly in Skidl. With my own tool, that would also be possible, as the ref dictionary files can be edited or prepared manually, so I can bypass the automatic annotation if needed.

Regarding hierarchy: even if it doesn't seem obvious at first, segmenting the design is again IMO mandatory if the circuit you're dealing with is non-trivial, and it has to be thought about in different ways than when considering hierarchical schematics. I'll give an example. One possible block in your case could be a "CPU block", containing obviously the CPU itself, but also its bypass capacitors, pull-ups, pull-downs, reset circuit. Anything that would logically be directly related to the CPU. Then another block could be "Memory", containing RAM and ROM for instance, with again their required bypass caps and any additional component directly related to them. Another possible block could then be an address decoding block, in which you would place all the required logic (and again the associated bypass caps). You get the idea. You'll see that it becomes manageable, and that you can reuse some of these blocks in another design later on.

Regarding errors, the main errors you can more easily make (compared to editing a schematic) is IMO missing connections and bad connections. ERC is sufficient for the first class of errors (as long as it can give you a complete list of unconnected ports/pins with the possibility of flagging ports that are deliberately NC (hope Skidl can do that) to avoid useless warnings. For bad connections, the risk is also there when dealing with a schematic. ERC can help flag gross errors such as shorts between power supplies, but obviously many can go undetected.

Since you're basically translating an existing schematic, if you have access to the original netlist, you can just compare the original netlist and the Skidl-generated netlist to check for inconsistencies. Otherwise, check again the output netlist against the schematic itself (a bit boring but it works).

Another class of errors I can think of is a missing part altogether. Again, with the output netlist, you should be able to check whether a part is missing. I think those uncorrect nets or parts errors are basically what can happen. Can't really think of other types of errors.

As for unit-testing, I'm not too sure how that could be done. Again correctness with a circuit (described with a schematic or with Skidl or similar tool) is IMO just about the right parts connected the right way. Only the netlist can help you pinpoint errors. Any automated testing would more or less require writing the expected circuit in a different form, which sounds a bit recursive.

To help verification on blocks, besides the global netlist, my tool can output local netlists (a netlist at each block level), so they are much easier to check by hand. See if Skidl can also do that: generate partial netlists, that helps a lot.

[obiwanjacobi]

Good points, thanks.

I guess it's ultimately a question of how to structure the code in a consistent, logical and readable way - in an attempt to make your code read like a book and therefor make it easy to spot any (obvious) mistakes. Not sure how realistic this goal is, though.

The example code I posted (2nd block) is of the 'CPU' part, with the Z80, RAM and ROM. Good point of adding all the surrounding elements (R's, C's) as well.

Not sure if SKiDL does not-connected errors, because I think I left some pins unconnected and don't believe I got any warnings. To be honest, I haven't worked on it for a while (work, family, life - you know the drill). But the author is very responsive to any suggestions so good chance we can get that in there if it's not already.

Unit testing: Yeah, I don't think writing code to check this type of code is very useful. I was looking for the certainty you get from unit tests in normal code - 'I know this part is working as it should'. I guess you could pull that up (in abstraction) to simulation, but that is a whole different ball game. Would be cool though, to declare you schematic in code and verify your design (and thereby the code) in simulation unit tests...

Manual verification: yeah I was thinking of printing out the schematic and crossing of / marking all the nets I have in my code by hand. But that is only possible when you have an existing schematic, which is usually not the case. So, how do you think about your EE design in your head and how does that translate to code? I think we're (at least I am) more used to a graphical picture, than a couple of pages of code. Also in communicating the design to other people, an image is more useful..?

This tool of yours you talk about, is that freely available?

[SiliconWizard]

I guess it's ultimately a question of how to structure the code in a consistent, logical and readable way - in an attempt to make your code read like a book and therefor make it easy to spot any (obvious) mistakes. Not sure how realistic this goal is, though.

Structure is key. Independent blocks. Now hoping to make the code as "readable" as a schematic - I'm not sure at all. Doesn't seem quite realistic to me. That's why I'm envisioning translating the circuit descriptions into graphical block diagrams. A graphical view helps tremendously, but you'd still keep the benefits of using "code".

Not sure if SKiDL does not-connected errors, because I think I left some pins unconnected and don't believe I got any warnings. To be honest, I haven't worked on it for a while (work, family, life - you know the drill). But the author is very responsive to any suggestions so good chance we can get that in there if it's not already.

If Skidl's ERC doesn't spot NC errors, you should definitely ask the author to add that. I wouldn't even consider code-based circuit design without it.

Unit testing: Yeah, I don't think writing code to check this type of code is very useful. I was looking for the certainty you get from unit tests in normal code - 'I know this part is working as it should'. I guess you could pull that up (in abstraction) to simulation, but that is a whole different ball game. Would be cool though, to declare you schematic in code and verify your design (and thereby the code) in simulation unit tests...

I guess that could be at least partly done. The blocks of a design that can be simulated could be translated to Spice/LTSpice circuits automatically, and then the simulations launched as automated testing, and the results compared to reference results. Why not. That could automate some of what we routinely do (or sometimes do not!) by hand.

So, how do you think about your EE design in your head and how does that translate to code? I think we're (at least I am) more used to a graphical picture, than a couple of pages of code. Also in communicating the design to other people, an image is more useful..?

It's very tough to do this from scratch without any graphical view. So I basically hand-draw partial/simplified schematics or just block diagrams and derive the code from that. I also use datasheets to help remembering what needs to be put around some ICs, etc.

Now once you have segmented the design properly, one good thing is that you can reuse some blocks in other designs. For instance, with my JTAG probe, there is a block around the FT2232H. I can reuse this block in any design requiring an FT2232H (the block includes the required caps, resistors, crystal, ...) Each block has ports, so it becomes a component as any other base component. You should definitely discuss with Skidl's author to figure out how to use hierarchical blocks, they are very useful. A completely flat design would be much harder to get right.

This tool of yours you talk about, is that freely available?

As I said above, it's not for now. I'm going to use it/improve it for a few more designs and I'll see what I do.

[obiwanjacobi]

How do you code sub-circuits? Is it a function that simply adds to a central (code) graph, it's parameters allowing customizations of some of the aspects it performs? Or is a function that builds a stand-alone (sub)circuit that then later gets added to the whole? Are you explicit in the input/output nets such a sub-circuit requires (hard params or properties on a class) -is this what you mean by ports?- or you just use a property-bag that contains named nets that need to be interfaced/connected?
Sub-circuits as components is a very powerful abstraction, I feel lies the solution. I'm just looking for the correct code constructs to make that usable/feasible.

SKiDL has Net.Fetch('<name>') that get-or-add's a net from the global pool (you see me doing that in the sample code in post #1). That is handy to connect stuff without pushing references all around your code. It does mean that the sub-circuit code determines the net name and that has to be matched to connect to it...

Visualization: I know SKiDL has a (plugin/3rd party?) graph visualizer, but that does not read as a schematic. As with all auto generated diagrams, positioning and layout are the trickiest to automate correctly.

Interesting food for thought...

[SiliconWizard]

How do you code sub-circuits? Is it a function that simply adds to a central (code) graph, it's parameters allowing customizations of some of the aspects it performs? Or is a function that builds a stand-alone (sub)circuit that then later gets added to the whole?

With my tool, sub-circuits can be, but are not necessarily built by functions. They are essentially objects. The basic "object" in Lua is a table, so I make heavy use of tables (which are pretty powerful).
So, a sub-circuit, that I called "module", is a structured table. You can manipulate it as any table, including making clones, etc. Essentially, a module is composed of: a list of ports, a list of nets, a list of sub-modules (interconnected by the nets at the module level), and additional attributes. What happens inside a module is completely local. Sub-modules can be any modules, so that's fully hierarchical. End-components are just modules that don't have sub-modules or nets.

Are you explicit in the input/output nets such a sub-circuit requires (hard params or properties on a class) -is this what you mean by ports?- or you just use a property-bag that contains named nets that need to be interfaced/connected?

Yes, each module has a list of "ports". A port is what can connect a module to other modules, has a name and can be a single-pin or a bus, and has properties such as direction (input, output, I/O, power in, power out, passive, ...) Those properties are used for ERC.

Sub-circuits as components is a very powerful abstraction, I feel lies the solution. I'm just looking for the correct code constructs to make that usable/feasible.

To me they are indeed. I suppose Skidl can allow that abstraction, but you should really ask the author now that maybe you have a clearer view of what you'd need.

Visualization: I know SKiDL has a (plugin/3rd party?) graph visualizer, but that does not read as a schematic. As with all auto generated diagrams, positioning and layout are the trickiest to automate correctly.

I'm curious what this plugin is using. I've tried using graphviz: https://graphviz.org/ , but it's not very adapted to this task. It supports nodes with "ports", which would more or less allow to render views similar to block diagrams, but when using ports, it doesn't support orthogonal connections, so it basically draws connections with curves. Really not pretty.

One option would be ELK: https://www.eclipse.org/elk/ , but I've never messed with Eclipse so far and am not skilled with Java. So I ruled that out for now.

Another option I found pretty interesting was netlistsvg: https://github.com/nturley/netlistsvg , seemed pretty promising but I'v tried a few things with it and it still has too many bugs to be really usable for this. Neat stuff though. As it's javascript (and actually uses elkjs), I'm not too keen on helping/modifying it myself either.