PyGTK, Multithreading and progress bar

Multithreading in Python seems to be very simple. Let’s look at the example:

import threading
import time

global_var = 0

class ThreadClass(threading.Thread):
	def run(self):
		global global_var
		while True:
			print global_var

class ThreadClass2(threading.Thread):
	def run(self):
		global global_var
		while True:
			time.sleep(1)
			global_var += 1

t1 = ThreadClass()
t1.start()

t2 = ThreadClass2()
t2.start()

t1.join()
t2.join()

Thread 1 is printing the global_var and Thread 2 is incrementing it by 1, every second.

Recently I needed to use threads in PyGTK to display progress bar. Off course the state of progress bar was dependent on the other thread. It had to be updated after each subtask call. This is the code:

import threading
import time
import gtk

def init_progressbar():
    main_box = gtk.VBox()
    progressbar = gtk.ProgressBar()    
    progressbar_box = gtk.HBox(False, 20)
    main_box.pack_start(progressbar_box, False, False, 20)
    progressbar_box.pack_start(progressbar)    
    info_box = gtk.VBox()
    main_box.pack_start(info_box, False, False, 10)    
    info_label = gtk.Label("Running...")
    info_box.pack_start(info_label)
    return main_box, progressbar, info_label


def run_tasks(pb, info_label):
    task_list = ['task1', 'task2', 'task3', 'task4']
    task_no = 0
    for task in task_list:
        pb.set_fraction(float(task_no)/len(task_list))
        run_task(2, task, info_label)        
        task_no += 1
    pb.set_fraction(float(task_no)/len(task_list))
    info_label.set_text('Finished')


def run_task(delay, task, info_label):
    info_label.set_text(task + ' is running...')
    time.sleep(delay)   # simulation of running the task


win_pb, pb, info_label = init_progressbar()
win = gtk.Window()
win.set_default_size(400,100)
win.add(win_pb)
win.show_all()
win.connect("destroy", lambda _: gtk.main_quit())
t = threading.Thread(target=run_tasks, args=(pb,info_label))
t.start()
gtk.main()

And…it was not working. The progress bar (and the label) did not get updated after task calls. Why?

It was caused by the fact, that PyGTK is not thread safe. Fortunately we have function threads_init() in gobject module, which can handle it. We just need to call gobject.threads_init() before first thread creation. Corrected (working) code looks like that:

import threading
import time
import gobject
import gtk

def init_progressbar():
    main_box = gtk.VBox()
    progressbar = gtk.ProgressBar()    
    progressbar_box = gtk.HBox(False, 20)
    main_box.pack_start(progressbar_box, False, False, 20)
    progressbar_box.pack_start(progressbar)    
    info_box = gtk.VBox()
    main_box.pack_start(info_box, False, False, 10)    
    info_label = gtk.Label("Running...")
    info_box.pack_start(info_label)
    return main_box, progressbar, info_label


def run_tasks(pb, info_label):
    task_list = ['task1', 'task2', 'task3', 'task4']
    task_no = 0
    gobject.threads_init()
    for task in task_list:
        pb.set_fraction(float(task_no)/len(task_list))
        run_task(2, task, info_label)        
        task_no += 1
    pb.set_fraction(float(task_no)/len(task_list))
    info_label.set_text('Finished')


def run_task(delay, task, info_label):
    info_label.set_text(task + ' is running...')
    time.sleep(delay)   # simulation of running the task


win_pb, pb, info_label = init_progressbar()
win = gtk.Window()
win.set_default_size(400,100)
win.add(win_pb)
win.show_all()
win.connect("destroy", lambda _: gtk.main_quit())
t = threading.Thread(target=run_tasks, args=(pb,info_label))
t.start()
gtk.main()

I wanted also to be able to cancel program computation, by the cancel button. To do that I needed shared variable to be a status flag informing, whether computation should be cancelled or continued. It had to be accessible in task loop and cancel event.

In Python there are mutable and immutable objects. Mutable are passed by reference, but immutable – by value. More about that on this StackOverflow question. Boolean type is immutable, but there is a way around this: use list (mutable type) with one element (boolean variable). I know it is awful solution, but I did not find any better. If you know one, let me know!

This is the app with cancel button:

import threading
import time
import gobject
import gtk

def init_progressbar():
    main_box = gtk.VBox()
    progressbar = gtk.ProgressBar()    
    progressbar_box = gtk.HBox(False, 20)
    main_box.pack_start(progressbar_box, False, False, 20)
    progressbar_box.pack_start(progressbar)    
    info_box = gtk.VBox()
    main_box.pack_start(info_box, False, False, 10)    
    info_label = gtk.Label("Running...")
    info_box.pack_start(info_label)
    cancel_box = gtk.HBox()
    info_box.pack_start(cancel_box)
    cancel_button = gtk.Button("Cancel")
    cancel = [False]
    cancel_button.connect("clicked", cancel_counting, info_label, cancel)
    cancel_box.pack_start(cancel_button, False, False, 20)
    return main_box, progressbar, info_label, cancel


def run_tasks(pb, info_label, cancel):
    task_list = ['task1', 'task2', 'task3', 'task4']
    task_no = 0
    gobject.threads_init()
    for task in task_list:
        if cancel[0]:       
            pb.set_fraction(0)
            info_label.set_text('Canceled')
            return
        pb.set_fraction(float(task_no)/len(task_list))
        run_task(2, task, info_label)        
        task_no += 1
    pb.set_fraction(float(task_no)/len(task_list))
    info_label.set_text('Finished')


def run_task(delay, task, info_label):
    info_label.set_text(task + ' is running...')
    time.sleep(delay)   # simulation of running the task


def cancel_counting(widget, info_label, cancel):
    cancel[0] = True
    info_label.set_text("Cancelling...")


win_pb, pb, info_label, cancel = init_progressbar()
win = gtk.Window()
win.set_default_size(400,100)
win.add(win_pb)
win.show_all()
win.connect("destroy", lambda _: gtk.main_quit())
t = threading.Thread(target=run_tasks, args=(pb,info_label, cancel))
t.start()
gtk.main()

To make above code working on your machine you need to have Python and PyGTK installed. To find out the details, how to install Python and/or PyGTK check my Python jump start post.


Why you should read “The Pragmatic Programmer”

The Pragmatic Programmer cover

Finally I read book written by Andrew Hunt and Dave Thomas: The Pragmatic Programmer. It was published in 1999, but it is still valid. The main idea provided by this book is the way of thinking. In other words: how ‘The Pragmatic Programmer’ should behave and what actions he should take in various situations.

There are also tips which skills are useful for programmer (e.g. using perl/bash scripts, mastering command line and at least one text editor).

Some advices are not valid in any case. E.g. in my opinion tip to never leave broken functionality has exceptions. Especially when it is some minor bug. There are situations when it is better to first focus on most important features and the problematic one leave for the end. After all we will have a few minor bugs and working main functionality. Instead of solved some or even only one, hard minor bug (like css styling:)) and not implemented main functionalities.

The other well-known (or should be well-known) tips like use Source Control, learn new technologies, analyze mistakes from past projects are also mentioned and described.

Here you can find list of tips from the book in nutshell: http://pragprog.com/the-pragmatic-programmer/extracts/tips. It is kind of summary of the book content.

At the and of each chapter, there is a set of exercises. This one blew my mind:

A quick reality check. Which of these “impossible” things can happen?

  1. A month with fewer than 28 days.
  2. stat(“.” ,&sb) == -1 (that is, can’t access the current directory)
  3. In C++: a=2;b=3; if (a+b!=5) exit(l);
  4. A triangle with an interior angle sum != 180°
  5. A minute that doesn’t have 60 seconds
  6. In Java: (a + 1) <= a

Fortunately at the end of the book there are answers (I wouldn’t even start googling, due to 100% certainty that some of them just can not happen, like 1 or 5):

  1. September, 1752 had only 19 days. This was done
    to synchronize calendars as part of the Gregorian
    Reformation.
  2. The directory could have been removed by another
    process, you might not have permission to read it,
    &sb might be invalid—you get the picture.
  3. We sneakily didn’t specify the types of a and b.
    Operator overloading might have defined +, =, or !
    = to have unexpected behavior. Also, a and b may
    be aliases for the same variable, so the second
    assignment will overwrite the value stored in the first.
  4. In non-Euclidean geometry, the sum of the angles of
    a triangle will not add up to 180°. Think of a triangle
    mapped on the surface of a sphere.
  5. Leap minutes may have 61 or 62 seconds.
  6. Overflow may leave the result of a + 1 negative (this
    can also happen in C and C++).

Summarizing: The Pragmatic Programmer contains huge amount of tips, which can help you to become better programmer. You may already know some of them (or most of them). In that case you will recall why they are important. Even Scott Hanselman said on his blog (in post Six Essential Language Agnostic Programming Books), that he ‘like to read this book at least every six months or so’.

It is 352 pages book. Can be easily read within 2 weeks by 1 hour reading per day. In my opinion it is one of must-read for programmer.


Research Assistant Job – Project Sireum

Since January 2013 I am Master student at Kansas State University. I am also Research Assistant at SAnToS lab.

Bakar toolset

Since the beginning I am working on project Sireum. It is focused on toolset for analyzing Spark/Ada programs: Bakar. Spark is subset of Ada programming language extended by code contracts. More info here. Bakar contains two main tools: Alir (information flow analysis tool) and Kiasan (verification tool for code contracts).

The Kiasan tool is based on symbolic execution. Simply saying: it checks all possible code paths and discover possible issues (like variable overflow, side effects etc.). More info about Spark contracts can be found in Tokeneer Discovery Tutorial. Comprehensive description of Kiasan in contained in this paper.

Kiasan plugin for GPS

My first task was to integrate Kiasan tool (Java app) with GPS (GNAT Programming Studio).

I have never working on IDE plugins before. It was something totally new for me. That’s why I was excited.

Plugins for GPS are written in PyGTK (Python graphic library). First I needed to learn Python and PyGTK. Then I followed GPS documentation (especially chapters 16 and 18).

The sample flow of running Kiasan plugin is as follows:

  • right click on method (which you want to analyze) and choose ‘Run Kiasan’
  • handle this event and get method name and package name (will be needed to run Kiasan)
  • run external Java program (Kiasan.jar) from Python code (it has command line interface, so I just needed to execute subprocess.call method), which generates JSON report
  • parse JSON report and display it in the PyGTK window attached to the GPS
  • add code highlighting, dependent on chosen method or case (in report window by double-click)

GPS along with Kiasan plugin is available in Sireum distribution, which can be downloaded from http://www.sireum.org/download.

In the demo below I am showing how to install Sireum distribution and GPS with Kiasan plugin on Windows.

By the way: Sireum Project is Open Source on github!

Do I like GRA job? Yes. It is very nice experience. I am able to work on Mac. I didn’t have this opportunity before (I wouldn’t buy it soon probably). I was able to learn Python (and PyGTK). I learn Design by Contract. I learn how to create plugin for IDE (GNAT Programming Studio). We have continues integration server – Jenkins: I learn how to manage it. I explore the World of ‘High Assurance Software’ and Ada programming language, which is most common in such environment. It is totally different than working as software developer in some company.

Recently I started working on second SAnToS’ project: Medical Device Coordination Framework. Another interesting thing. But I will tell you about it in another post.

Interesting fact: Ada programming language was named after Ada Lovelace (1815–1852), who is credited as being the first computer programmer.


Quick look at Java Spring MVC framework

Java software market is still bigger than .NET. Many times, when I heard that some friend on mine is working as Java Web Developer, he is using Spring MVC framework. Finally I decided to look at this (most popular?) Java Web Framework.

I started with Pluralsight.net tutorials: Introduction to Spring MVC and Spring with JPA and Hibernate.

First surprise for me was a lot of xml based configuration, which needs to be written by developer. E.g. we need to configure which class is controller by declaring its path in xml file (servlet-config.xml). For some other elements we need to provide long uris as packages’ names. So easy to misspell. This is ‘configuration over convention’, instead of ‘convention over configuration’ as it is in Ruby on Rails or ASP.NET MVC.

As an example, look at this piece of xml, which is needed to handle JSON and XML requests (all written by hand):

<bean class="org.springframework.web.servlet.view.ContentNegotiatingViewResolver">
		<property name="order" value="1" />
		<property name="contentNegotiationManager">
			<bean class="org.springframework.web.accept.ContentNegotiationManager">
				<constructor-arg>
					<bean class="org.springframework.web.accept.PathExtensionContentNegotiationStrategy">
						<constructor-arg>
							<map>
								<entry key="json" value="application/json" />
								<entry key="xml" value="application/xml" />
							</map>
						</constructor-arg>
					</bean>
				</constructor-arg>
			</bean>
		</property>
		
		<property name="defaultViews">
			<list>
				<bean class="org.springframework.web.servlet.view.json.MappingJacksonJsonView" />
				<bean class="org.springframework.web.servlet.view.xml.MarshallingView">
					<constructor-arg>
						<bean class="org.springframework.oxm.xstream.XStreamMarshaller">
							<property name="autodetectAnnotations" value="true" />
						</bean>
					</constructor-arg>
				</bean>
			</list>
		</property>		
	</bean>

Going back to controller: besides its configuration in XML file, we need to add annotation @Controller on its class and @RequestMapping on methods (no default routing). Additionally we always need to return the name of view (instead of some default value). Here is a sample controller with one action:

@Controller
public class HelloWorldController {

    @RequestMapping("/helloWorld")
    public String helloWorld(Model model) {
        model.addAttribute("message", "Hello World!");
        return "helloWorld";
    }
}

Above code will return the view helloWorld.jsp (according to convention, located in webapp/WEB-INF/jsp/ directory):

<!DOCTYPE html>
<html>
    <head></head>
    <body>
        Message: ${message}
    </body>
</html>

The result should looks like that:

Spring MVC Hello World

Connecting application with database is not easier. Adding dependencies and writing code to make Hibernate working with the app took 1h(!) in Pluralsight tutorial (Spring with JPA and Hibernate). It is over 50 lines of xml. Additionally as we know – it is very easy to make a mistake during writing xml. In ASP.NET MVC it can be done automatically (eventually we can change data source by specifying connection string). I didn’t find easier way to do that (in Spring). Of course we can use copy/paste method and just set some properties, but…where is code generation for that?

After all, I would like to notice how big step was made by other MVC Web Frameworks like Rails, Django or ASP.NET MVC. Lot of stuff, which has to be written by hand in Spring, is already implemented under framework layer (in Rails, Django and ASP.NET MVC). However Spring MVC was also big step ahead, when it was created in 2002. In this video people are explaining that (e.g. because of Spring they do not need to rewrite lots of boilerplate code anymore).

I wonder whether next version of Spring MVC will have more ‘conventions’ and default configurations. It will make the framework more developer friendly and easier to maintain.

If you want to start with Spring MVC i recommend you to start with Pluralsight.net tutorials: Introduction to Spring MVC and Spring with JPA and Hibernate. You can also find this website useful, as well as Spring Framework Reference Documentation. I didn’t read any book, but I found those two most often recommended: Spring Recipes and Spring in Action.


The Ten-Day MBA

The Ten-Day MBA cover

Chad Fowler, in his book “The Passionate Programmer” recommended a book The Ten-Day MBA. The reason, why he recommend it, is the fact that successful programmer should know the basics of marketing. E.g. to be aware how much his work is worth.

The Ten-Day MBA is an overview of fundamental basics of economy, business, finance, leadership and even real estate. In my opinion, this book is not only for programmers. It is for everyone! You can find there difference between monetarists and Keynesian economy, the product life cycle (from factory to customer), the difference between corporation and private business, time estimation of projects, business strategies etc.

I strongly recommend this book. I was studying at University of Economics 3 years (Bachelor degree). This book contains all most important concepts I learned there.

You can find it on amazon.