access method

By Guanqun on February 4, 2010 10:05 PM | No Comments | No TrackBacks
In Stonebraker's "The Implementation of Postgres" paper, he et al mentioned that they designed the access method TOO HARD. Because there were two many connections for this sub-system to work well. You have to know:
  1. lock manager
  2. buffer manager
  3. know the states between the query engine
  4. write 13 non-trival routines
So it seems it's a good start to get hands dirty on this part, hopefully it will get myself quicker to be familiar with this database system.

connect to postgresql on another machine

By Guanqun on February 4, 2010 10:23 AM | No Comments | No TrackBacks
If the postgresql is setup on a server, but you want to connect to it on another machine, say on your windows machine. We have to modify the pg_hba.conf file in DATADIR directory, add one line to it: 
host    all     all             10.0.0.0/8              trust
if the postgresql server is running, you can invoke 
pg_ctl reload -D DATADIR
to reload the configurations. HTH
Continue reading connect to postgresql on another machine.

'busy loop' vs 'sleep' related functions

By Guanqun on January 29, 2010 6:30 PM | No Comments | No TrackBacks
If you're using usleep() or such of sleep functions in your program in order to mimic the latency, then you should be aware of the fact that it is VERY inaccurate. The smaller it sleeps, the more it becomes inaccurate. Why? Because with sleep, it will introduce more context switches.To accommodate this program, we can take advantage of 'busy loop'. This technique is widely used in embedded system. It's simple, ugly? or effective? As long as it works, it's good!

Migrate to VMware Fusion

By Guanqun on January 27, 2010 8:55 PM | No Comments | No TrackBacks
Last night, I spent about two hours to migrate my PC to a virtual machine in VMware Fusion on my MacBook Pro. It was faster than I expected. The progress bar read about three to four hours, but since I connect these two machine on a cable, it was very fast to finish the process.

The migration failed actually on my first trial, the progress bar stopped at about 29% and then no more progresses any more. To be safe, I stopped the firewall on my Windows PC, and I disabled the screensaver as well.

After about one hours' anxious waiting, everything works fine on my Mac, expect it runs slower. 

Hmm, I'm thinking making my MacBook Proc 4G memory.

implicit declaration 'pwrite64'

By Guanqun on January 27, 2010 2:15 PM | No Comments | No TrackBacks
I was re-factoring a library when I encountered a problem complaining that "implicit declaration 'pwrite64'." It can be ignored, but programmers should not ignore every single 'warning'.

After some digging, I find that it's easily fixed by adding "#define _GNU_SOURCE".

notice the size of type

By Guanqun on January 22, 2010 5:47 PM | No Comments | No TrackBacks

I was manipulating a large number of arrays, and when I pushed some item into it. I increase the index one at a time. But note that the array is huge. But poor me, I was trying to write '

int32
' but instead, I wrote '
int8
'. So you can image that this variable is overflowed and then wrapped back. Note the size of the type, I'm telling to myself...

'free' problem

By Guanqun on January 22, 2010 5:36 PM | No Comments | No TrackBacks

I was modifying a large program, and I was going to change an interface in order to reduce the data copy.

+--+ <-- allocated outside foo()
|  |
+--+ <-- passed to foo(), try to free this address.
|  |
|  |
|  |
+--+

The actual space is allocated outside the function foo(), and as I change the interface, I only pass the second part of the space to foo(), but I forgot to modify foo() as well, therefore, foo() still thinks the parameter passed in may be free()'ed. Then oops!

monitor the cpu usage

By Guanqun on January 22, 2010 2:07 PM | No Comments | No TrackBacks
Checking the CPU utilization is a common task in development and testing. Because we'll be interested in how the CPU resources are used. There are several ways to get this information.

  1. 'cat /proc/loadavg'. This command line will give you an overall picture of how the system is running. It spans a long time.
  2. 'top'. This shows you some text graphics displaying how each process is playing in the system.
  3. 'htop'. It's an enhanced tool of 'top'. This tool takes advantage of ncurse library. And its output is really nice for boring administrators.

But what about when you're trying to calculate the CPU usage in your C/C++ program or you want to monitor the CPU usage every second. Therefore it will give you a nice picture finally.

Let's check out my little Python cpu-usage.py. It reads the file '/proc/stat' and calculates the percentage of how many jiffies are used in user computation. It may not be very precise, but it's enough for general use.

Hope you'll enjoy this little tool.
Continue reading monitor the cpu usage.

Centos 5.4 saves my life

By Guanqun on January 18, 2010 11:00 PM | No Comments | No TrackBacks

We got an DELL XPS machine and we need to set it up. However, there are several restrictions right now. The main obstacle is that one Huawei card (SSD, PCI-e slot) is installed but their company only provides us with a 2.6.18 driver. That's said, we have to installed it on 2.6.18 kernels.

There are RHEL5u1, 5u2 ... out there. I tried with RHEL 5u1, but it can't boot due to lack of proper disk driver. Duh.

Centos 5.4 saves my life. I'm afraid RHEL5u4 should work as well, but it's hard to get. This machine boots well: the disk driver(ahci) is OK, the network driver(tg3) is OK.

However, there's a small itch to scratch. The given SSD driver is of rpm format. And by default with Centos 5.4's kernel version, it will fail due to some dependency check (e.g. it requires kernel version should be less than el5.94). To accommodate this problem, I extracted this rpm package via this command line:

rpm2cpio ssd.rpm | cpio -idmv

and then manually install the kernel module. It works as expected though it outputs some message regarding this module will taints the kernel. It doesn't matter however as long as it works fine.

wget with no passive ftp

By Guanqun on January 17, 2010 2:35 PM | No Comments | No TrackBacks
By default, wget will invoke with passive mode for FTP retrieval. Generally speaking, it's the only way you can play with FTP behind an firewall. However, there is no exception for no rule. I met with this problem in my corp. I have to invoke "wget --no-passive-ftp <ftp-url>" to retrieve the stuff, which is the first time I've encountered.
Archives