Migration to Python 3 and virtualenv

So far Python Power Electronics has been built on Python2. I have Python 2.7.9 on computer and most of the other versions of Python I have used are quite similar i.e Python 2.7.x. Python 2 is now legacy and support will be disabled from 2020. Though that is a couple of years away, I figured I should start migrating since Python 3 is now well established and almost every OS ships with Python 3 by default and it may be soon when installing Python 2 may actually become an issue.

To start this migration, I don't want to do a system wide installation of Python 3 as it will break all my projects. So I am starting with virtualenv. To begin with, I am installing Python 3.5.4 into a separate folder.

In my Linux system, the system wide packages are installed in /usr. So I am installing Python 3.5.4 in home:
/home/user/python_3_5_4
In the above user is the name of the user that you are logged in as. For that matter, you could install this in any other folder of your choice. I would strongly suggest that you install Python 3.5.4 in a user writable directory and not in a root directory.

So, download Python 3.5.4 from the Python website:
https://www.python.org/downloads/source/

So let us suppose, this is downloaded to:
/home/user/Downloads
Extract the zip file to:
/home/user/Python-3.5.4/
Change into this directory. In this directory you will find the entire source.

There will be some dependency issues - something is missing etc. I had a problem with libssl-dev being missing. I installed it using apt-get on my Debian system.

Anyway, first compile the source:
/home/user/Python-3.5.4/ $ ./confiure --prefix=/home/user/python_3_5_4

The prefix states that Python should be installed in a special folder and not in /usr/local/ which is the default.
If you get any errors and you might, Google each error and check which package is needed. Quite often it may say a package is missing when it is installed. What might actually missing is the development package. For example libssl may be installed, but libssl-dev may be missing. But that may cause it to exit with an error.

After that run the command:
/home/user/Python-3.5.4/ $ make

And if no errors:
/home/user/Python-3.5.4/ $  make install

The last command will create within /home/user/python_3_5_4 separate bin/, lib/ etc directories where Python will be installed.

Now, if you check:
python -V
You should get Python 2.7.x.

But if you run:
/home/user/python_3_5_4/bin/python3 -V
You should get Python 3.5.4.

So this means Python 3.5.4 has been installed in a separate directory. Now to begin using it.

Create a directory in home or anywhere else which will be the separate environment for Python 3.5.4. For example,
/home/user/python_3_5_4_virtual

Change into this directory. Now you need virtualenv installed. I didn't have it so I used apt-get again to install virtualenv.

Run the command:
/home/user/python_3_5_4_virtual $ virtualenv -p /home/user/python_3_5_4/bin/python3 python3_install

What this will do it create a virtual environment but will use this Python 3.5.4 installed in an isolated directory to create an install environment in the directory called python3_install. So python3_install will contain all software related to Python - django, matplotlib, numpy, scipy etc.

Once this is done, activate it:
/home/user/python_3_5_4_virtual $ source python3_install/bin/activate

Now a special session will start. If within this session, you run:
(python3_install) /home/user/python_3_5_4_virtual $ python -V
You will get Python 3.5.4. The (python3_install) in bracket means you are in a virtual environment.

If you exit this environment by running:
(python3_install) /home/user/python_3_5_4_virtual $ deactivate

You exit. Which means:
/home/user/python_3_5_4_virtual $ python -V
Gives Python 2.7.9. Notice that (python3_install) has disappeared meaning you have exited the virtual environment.

So, to perform the migration, I will enter this virtual environment, install dependencies with Python 3.5.4 as the base. I will check the operation of the simulator bit by bit.

Tutorial on PV panel simulation

I released a tutorial on how to simulate a PV panel.
http://pythonpowerelectronics.com/tutorials/tutorial3/post.html

This is the first step towards simulating the interaction of renewable energy systems with the grid and also on how to build smart grids and microgrids with renewable energy. In the past I have simulated converters for interfacing PV to the grid and I have usually assumed the PV panel as a dc source. As I was focused on the control of the power converter, assuming the PV panel as a dc source, was not a problem. But if I need to simulate how PV can be used in any system, I need a more accurate model of a PV panel. In the least, I need to be able to replicate the V-I characteristics of the PV panel. It would be a good addition if I could also model how the output of the PV panel is affected by radiation, temperature etc.

The problem is that a lot of the papers that deal with PV are pretty heavy into solid state physics. The widely popular current-source with anti-parallel diode equivalent circuit for a PV panel is usually used. However, where most of the papers defer is how to generate the constants that the current generated and drawn by the diode are expressed with respect to. The only data available from a PV panel specification are of limited use when it comes to solving these equations. In order to be able to completely express the equations, constants have to be determined by solving non-linear equations.

My objective was to generate a PV model that was reasonably accurate and also understandable to a power electronics engineer without an in-depth understanding of device physics. And I found that several approximations were possible to determine many of the constants that appeared in the equations.

The most important assumptions were around the two extreme operating points - open-circuit and short-circuit. At short-circuit, the entire photo current generated flows into the short. The short-circuit current at a reference temperature is provided in the datasheet. Therefore, the photo current can be equated to the short-circuit current. Conversely, at open-circuit, the entire photo current flows into the anti-parallel diode. Therefore, the diode current is now equal to the photo-current which at the reference temperature is equal to the short-circuit current. The concept is that the value of the photo current generated depends only on the temperature and solar radiation. At short-circuit, this current flows into the short while at open-circuit, it flows into the diode.

The remaining calculations are merely procedure. Finally, the PV current that has been calculated is modeled as a voltage behind a resistance. A capacitance is connected across the PV terminals to stabilize the voltage.

Check out the report in the tutorial link for more information.

Launching the tutorial series with PLL

I am starting a series of tutorials on topics in power electronics and power systems. The purpose of these tutorials is to provide an educational resource for practicing engineers. The tutorials are made as simple and visual as possible and uses simulation to demonstrate concepts. Mathematical equations are used when necessary but every attempt is made to NOT make the tutorial mathematically rigorous so that the report is an easy read.

The first tutorial is on PLL and can be found in the link:
http://www.pythonpowerelectronics.com/tutorials/tutorial1/post.html

The report is available for free but the simulation examples have to be purchased from the e-commerce website Gumroad:
https://gumroad.com/l/HGWQ#

If all you need is a brief idea about PLL, the report is all you need. But if you would like to follow the tutorial step-by-step, the simulation examples will help you tinker with control gains and other parameters.

The objective behind the tutorials is to use the flexibility and convenience of simulations in understanding complex problems in advanced power systems. In this tutorial, you learn how grid synchronization is important and what happens without it, how to design a PLL for single and three-phase systems. The simulation files are just a starting point. You could always use them to try out some additional control.

Command line versus web application

The development of the web application for the circuit simulator is under way and can be found on my GitHub account:
https://github.com/shivkiyer/circuit-simulator/tree/sim_with_django

For a while, both command line and the web app will be available simultaneously for those who would prefer command line. The main disadvantage of the web application is the need to have Django installed. Though not a significant task, for someone who doesn't want a server running on their computer, it may not be the best option. I am still thinking of ways to automate the setup of the Django server so that maybe a single script could have everything up and going.

The first few steps in this direction seem promising. For example, so far, I have been able to replace the circuit_inputs.csv file. When a new simulation is created, the first form is the one that contains the fields similar to circuit_inputs.csv. Django provides features where error messages can be automatically generated when the user enters wrong data. For example, the user needs to specify the working directory that will contain the circuit schematic spreadsheets and other files. This working directory will be used also to write the output data file and therefore needs to have write permission to it. Since this Django app should be preferably run as user rather than root or Administrator, the working directory chosen should be writable by a user. This check is performed at the model level where a validation method has been written. This validation method writes a dummy file just to check if the app can write into the directory and failure to do so results in an error which shows up in the form. So the user can't submit a form without choosing an appropriate directory.

The user interface is divided into several segments. The first form above will only ask for basic parameters like time duration, time step, data storage interval, working directory etc. Once done, these parameters will appear as a table and the next form will ask for the user to choose the circuit spreadsheets from a file browser form. The user can still go back and edit the simulation parameters with an Edit parameter option. The file browser button makes life easier in choosing files rather than write them in circuit_inputs.csv. It would also be possible to check if the files are .csv files an also if they actually exist in the working directory.

The same goes for control files. They will have similar file browser buttons that can allow the user to choose Python control code.

As compared to the command line, the major advantage is that errors are pointed out right away and in a little more friendlier manner. The command line does provide error messages and I will try to bring it close to the web app in terms of how immediately the errors are pointed out. This might save the user progressing to the last stage and then seeing the simulator point out an error in the schematic which should have been picked up right away after the circuit_inputs.csv file was provided.

In the web app, errors in the circuit file and also errors in parameters can be generated by processing all these files. The same checks could be imposed - paired jump labels, closed branches etc. Since, a circuit can only really be checked after all circuit spreadsheets have been read and processed, this error generation will need to take place at a higher level.

What will be challenging is to generate parameters for all the circuit spreadsheets. The simplest way to do this will be create models for every component and add them to the circuit files as ManyToMany objects. This also allows a ModelForm to be created to every component model and further to connect each component to the circuit spreadsheet. Eventually, the parameters obtained from the ModelForms will need to be returned to the classes in circuit_elements. Which means the read_parameter methods of every class will need to be changed or overloaded for an interface to the web app.

Django seems to provide a fairly convenient and hierarchical structure.

Simulation Case  ---> Circuit files ---> Component objects
                            ---> Control files ---> Control objects

User interface

One of the biggest drawbacks of the circuit simulator has been the lack of a graphical user interface. For a Linux user, running a software from a command line is not too much of a hassle, but for users of other operating systems, it usually is. A graphical user interface where one or a few windows gives you everything you need is what everyone wants. Something like that may be too much to build at this point of time, but I have been considering options that can provide a consolidated interface without a full-scale graphical component.

Since the time I have been using Django for web development, I have been thinking of using it to create a user interface for the simulator. Django is fairly easy to install in any operating system. It is fairly easy to setup particularly as a standalone server with the SQLite database good enough for development applications and also easy to configure. The database setup scripts are not too hard. And once the server is running, you could access the app through a web browser.

There are aspects of the simulator that are good the way they are. For example, designing circuits through spreadsheets. That will remain for a long time as it is fairly convenient and also spreadsheet software are available for any operating system. So trying to replace the design of the circuit at the current moment is not really necessary. However, many other aspects are. For example, entering the circuit parameters. This could be made much more interactive through a web app. Once the circuit design is processed, instead of asking the user to change the parameters in the spreadsheet, an interactive form can be designed to help the user out with entering circuit data. The next is in designing controllers. Right now, every controller has a descriptor spreadsheet which lists the inputs, outputs and also special variables. This process can be made interactive as the user can be allowed to choose meters from any spreadsheet or outputs from any sheet. The last and the most important is the plotting of circuit data. Using Gnuplot is extremely convenient but for most this just isn't interactive enough. If a plotting interface can be built into the web app where a user can choose the x and y axis of every plot and simply click a button to get the plot either embedded in the web browser or as a separate window, it would be much simpler. Eventually, the result is what is most important to the user, and I need to make the viewing of results a little easier.

Since this simulator is for large circuits with a large amount of data being generated, trying to run the simulator remotely over a server is not a feasible option. It may be for smaller circuits that do not require much data being transmitted over the network but for larger circuits, it wouldn't work. So the purpose of using Django is merely to build an interface that is almost graphical but without using something like Tkinter or PyQt that would take me a lot of time.

Django allows the user to enter data in forms similar to any web form. The only additional layer is to make the forms relevant to the circuit by creating a model layer. Moreover, this model layer will have to be dynamic as every circuit is unique. Therefore, Django will merely replace the cells in a spreadsheet with HTML forms that hook up to a database.

Releasing the book

I am happy to announce the release of my book "Simulating non-linear circuits with Python Power Electronics: an open source simulator based on Python". The book has been self-published and released on Gumroad. The link is:
https://gumroad.com/l/lYQK#
To read a sample chapter, I have released Chapter 4 of the book on my website:
http://www.pythonpowerelectronics.com/papers/chap4.pdf
Follow my Facebook page for continuous updates:
http://www.facebook.com/pythonpowerelectronics

The outline of the book is below:
----------------------------------------------------------------------------------------------------------------------------

Chapter 1 introduces the concept of simulation and describes the challenges in circuit simulation. Python Power Electronics as an open source circuit simulator is proposed with its objectives and target audience.

Chapter 2 provides an overview of the Python programming language. For a detailed tutorial, a reader is recommended to either read a book on Python programming or follow an online tutorial on the internet. The purpose of the chapter is to enable the reader to understand the code segments that will be provided in the subsequent chapters that deal with user-defined control functions and the case study.

Chapter 3 describes the interface that the simulator uses to interact with the user. The chapter describes the philosophy behind choosing spreadsheets as the mode of extracting information from the user. Spreadsheets are used by the user to enter simulation parameters,circuit schematics, parameters of the components in the circuit schematics and also the structure of control functions. The chapter describes how the structure of every component class in the simulator library and how the data entered by the user is processed by each component class. The chapter also describes the concept of how classes are instantiated for every component found resulting in objects and how these objects are referenced by the simulator. The chapter describes the execution flow in the simulator and how the simulator processes the data provided by the user and makes it available to the core simulation engine. The chapter does not describe how user-defined control functions are processed as the whole of Chapter 4 is dedicated for this purpose.

Chapter 4 describes how a user can write control functions for a simulation. Chapter 3 has described which of the circuit components can be controlled externally. Besides these controllable components, a control function need not perform a control action, but can instead be used to process simulation data or perform calculations. The chapter describes how the control functions have to be written as Python 2 files and specified in the simulation parameter spreadsheet. Every control function will have an interface to the simulation in terms of inputs and outputs and this interface is described by a spreadsheet called a descriptor. Besides inputs and outputs, every control function can use certain types of variables that perform special functions. The chapter describes the importance of each type of control variable and how they are implemented in the simulator. The chapter describes how control functions are scheduled by the simulator using time events and with an example, it is described how the simulator ensures that the control functions execute at the desired time instant. A simple example has been provided to describe how control functions can be interfaced with the simulation and also with each other.

Chapter 5 describes how a user can simulate a circuit with a power electronic converter. The example chosen has been a shunt connected three-phase VAR compensator realized using a two-level voltage source converter in a three-phase system. The voltage source converter consists of controllable ideal switches that are turned on and turned off by pulse width modulation. The chapter describes how the user can build this simulation in stages such that every new subsystem added to the circuit can be verified. The chapter also describes how the user can write control functions with detailed examples of each control function in the simulation and also design the control interfaces through descriptors. Every stage of the chapter contains simulation results to show how the project develops. Through this example, every feature of the simulator has been described with details so that users can develop their own simulations.

Chapter 6 describes how the simulator processes the circuit schematics that the user enters in spreadsheets. The connectivity information is extracted from the circuit schematics in the form of nodes, branches and loops. Nodes, branches and loops are used to perform circuit analysis through loop analysis and nodal analysis which are described in the next chapters. The chapter describes through sample circuits, the algorithms used to determine the nodes, branches and loops. The chapter introduces the concept of the LoopMap which is used for performing loop analysis in Chapter 7 and the concept of KCLBranchMap which is used for performing nodal analysis in Chapter 8.

Chapter 7 describes how loop analysis is performed in the simulator. The chapter describes how the matrix equation for performing loop analysis is generated from the LoopMap described in Chapter 6. A brief description is provided about how the matrices in this equation are transformed using row operations such that they can be solved by using numerical integration techniques. The chapter describes how loop currents and branch currents in the circuit can be mapped which allows for calculation of branch currents from loop currents and vice versa. The chapter describes with an example how time constants of branches of the circuit can make the simulation unstable and introduces the concept of a stiff loop. By providing a sample circuit and its corresponding LoopMap, the chapter describes the need to isolate stiff loops so as to be able to simulate a circuit. With this example, the concept of loop manipulations is described and with advanced examples, the effectiveness of the procedure is described. The chapter describes the limitation of loop analysis with another set of examples and therefore the need for nodal analysis.

Chapter 8 describes how nodal analysis can be used to determine the currents through stiff branches (that have a very low time constant) in the circuit. With the example of a simple buck converter, the chapter describes how loop analysis is insufficient in determining the conduction of power devices during switching events. The chapter then describes how nodal analysis can be used effectively in determining how power devices conduct and the transfer of current from one device to another. The chapter introduces the concept of events and how the matrix equations for the circuit will be constant until an event occurs. The chapter finally describes the logical flow of processes in the simulator as it performs loop analysis and nodal analysis one after the other.

Chapter 9 will conclude the book by highlighting the advantages of the simulator and the future development intended in this project.

The book

Hope everyone had a good new year celebration.

This January, my goal is to complete the book for publication in February. So that means simulation of new circuits will come to a temporary stop. The first revision of the book is ready and for the table of contents check out:
http://pythonpowerelectronics.com/documentdownloads.html

A rigorous proof read is currently in progress and should be complete in two weeks. After that, another two weeks will be spent in editing the book, arranging the layout and ensuring a printable camera ready format.

The book will be self-published on Amazon using CreateSpace where they will print copies of the book on demand. Besides this I am also looking at other avenues where a simple PDF can be made available for those not wanting to buy a printed book but wanting simulation tutorials and code along with the pdf of the book. Still need to work out the details of that but that will probably follow only after the book has been released.

As for the book, I will be providing one chapter of the book on my site for download so that anyone can get a flavour of the book. In the meantime, check out the short papers that have been uploaded on the above link. For regular updates, follow my facebook page:

http://www.facebook.com/pythonpowerelectronics/

Let's hope for a good 2017!