Kurt's Weblog

Logging Libraries for C++

Which one should I use? I would like to be able to set a compile flag to complete remove the logger from a build (ala NDEBUG). Is this possible?

I am leaning towards log4cxx at this point, but I don't have a whole lot of info on this.

Continuations and advanced flow control from IBM.

I still need to learn python generators and decorators.

moodle

  Moodle is a course management system (CMS) - a free, Open Source
  software package designed using sound pedagogical principles, to
  help educators create effective online learning communities. You can
  download and use it on any computer you have handy (including
  webhosts), yet it can scale from a single-teacher site to a
  50,000-student University. This site itself is created using Moodle,
  so check out the Moodle Demonstration Courses or read the latest
  Moodle Buzz.

Installing Moodle

How to find devices on your local network

Turns out you can ping the broadcast address and most machines on a subnet will respond. The first trick is to find the broadcast address. Run "ifconfig". Look for the "broadcast" keyword followed by an address that probably ends in 255. For example:

$ ifconfig
en1: flags=8863 mtu 1500
        inet6 fe80::20d:93ff:fef0:6111%en1 prefixlen 64 scopeid 0x5 
        inet 192.168.2.101 netmask 0xffffff00 broadcast 192.168.2.255
        ether 00:0d:93:f0:61:11 
        media: autoselect status: active
        supported media: autoselect

Based on the 3rd line, the ping should be

$ ping 192.168.2.255
PING 192.168.2.255 (192.168.2.255): 56 data bytes
64 bytes from 192.168.2.101: icmp_seq=0 ttl=64 time=0.285 ms
64 bytes from 192.168.2.1: icmp_seq=0 ttl=64 time=1.615 ms (DUP!)

Then use "arp -a" to see the IP addresses that the computer knows about. For example:

$ arp -a
onefish.ucsd.edu (132.239.80.86) at 0:9:e9:1c:78:80 on en0 [ethernet]
sio-pps-rh-ptr.ucsd.edu (132.239.121.4) at 0:0:f0:a5:74:f2 on en0 [ethernet]
squidward.ucsd.edu (132.239.121.23) at 0:2:55:7a:b5:2e on en0 [ethernet]
...

A comment also suggests something like this "nmap -sP 192.168.1.1/24". Be careful with nmap so that you don't get yourself in trouble.

Some follow on about broadcasting and multicasting: Network Broadcasting and Multicasting from http://www.comptechdoc.org/

Grr. This is the 2nd time in the last couple hours that an openGL X11 app has caused a kernel panic on this Quad G5 tower. This program should not really be stressing the system. Where can I find out what is causing the kernel panic? I know the app has bugs, but a kernel panic? I would expect it just to crash the program.

A while back I was talking with Drew at SIO. He uses and likes plone for a lot of the stuff that he does. I really do not know much about plone and it is on my list of technologies that I would like to know more about.

Real-world Plone Experiences

Perhaps this will fix my code injection issue that I wrote into a program last month. Calling eval on the text in an XML file is scary. I have not read this closely enough to know if it really will solve my problem.

Restricted "safe" eval in python.

Looks like an interesting discussion

How To Make A Map: Image to Map

SQLAlchemy world domination tour

Here is a little something that I picked up from Alex a couple of months ago, but only now put into use. When doing inheritance in python, people usually do an explicit call to the parent's __init__ method like this:

class a:
  def __init__(self):
    self.foo='a'
class b(a):
  def __init__(self):
    a.__init__(self)

That works, but there is what I think is a better way. What if the inheritance tree changes? It is up to the programmer to go in and change every refernce to the base (aka parent) class to the new base class. Here is the method that Alex told me about:

#!/usr/bin/env python
class foo:
    def __init__(self):
	self.x=1 
class bar:
    def __init__(self):
	self.y=2
class foobar(foo,bar):
    def __init__(self):
	self.__class__.__bases__[0].__init__(self)
	#self.__class__.__bases__[1].__init__(self)
	self.z=3
##############################
f = foobar()
#
for i in range(len(f.__class__.__bases__)):
    print 'base['+str(i)+'] is ',f.__class__.__bases__[i]
#
print 'f.x',f.x
print 'f.y',f.y

The key is to call every base __init__ method so that the local world is setup correctly. I have commented out the 2nd init to show that by missing one __init__ from the parents, the child is damaged goods:

./bar.py
base[0] is  __main__.foo
base[1] is  __main__.bar
f.x 1
f.y
Traceback (most recent call last):
  File "./bar.py", line 20, in ?
    print 'f.y',f.y
AttributeError: foobar instance has no attribute 'y'

This sounds very fishy (no pun intended). Both are from seacoastonline:

Using the power of current

  PORTSMOUTH - A proposal to capture electricity from tidal energy in
  the Piscataqua River has been filed with federal energy officials
  and calls for 50 to 100 underwater turbines to be submerged
  throughout Rockingham and Strafford County stretches of the river
  and up into York County.



  According to the Federal Energy Regulatory Commission, the
  "Portsmouth Area Tidal Hydroelectric Project" would generate 8.76
  gigawatt hours from each of the units annually, and the power would
  be sold to an unidentified "local utility."
...

Here is a more skeptical view: Engineer criticizes hydro plan...

  Rob Cinq-Mars calls the proposed Piscataqua River hydroelectric
  project "baloney."  

Tidal energy is a good thing, but it sounds like whoever is filing this kind of blanked application all across the country is trying a get rich quick scheme. Wave and tidal power can have all kinds of problems and must be carefully planned.

Writing some tests for {32,64}-bit code made me wonder if it is time to build a 64-bit only tree for fink. I believe that the a good number of the packages will not work as the Apple 64-bit web pages imply that anything to do with graphics and user interface must be 32-bit. I don't see a /usr/X11R6/lib64 or equivalent, so my hopes are not high for X11 based programs. However, it would be great to have access to, for example, a 64-bit GMT so that it is possible to work with grids that are larger than 2 or 4 GB. I am not really sure what the limit actually is with the 32-bit Mac API. Last year I got errors when trying to mmap files larger than 2GB. Perhaps other calls can handle files up to 4GB, but I don't know one way or the other.

I was talking to Walter Smith last week. The conversation kicked off with a discussion on the topic of whether or not to install the new Code 2.3 dmg from Apple (the answer turned out to be NOT YET). Walter and I were discussing a number of other coding issues and he brought up the question of what is the proper way to do 32/64 size_t handling in printf that does not generate warnings. The key part of the this for GMT (the package in question) is that Walter and Paul decided way back when that GMT should be restricted to Posix. The reason being portability. If a call or parameter is not in Posix, then it is likely to not exist for some platform somewhere. I gave the problem an hour or two of research and found that the current state of the world is that size_t in posix will be trouble. They will have to do a cast and live with it. If the requirement of strict Posix is removed, c99 turns out to have a solution to this problem.

The following are excepts from my suggestions (if c99 is allowed):

I think (but am not 100% sure) that I have figured out the ```correct''' way to handle printf with size_t and handling 32/64 builds. This solution "works for me" and seems to match "man 3 printf" on Mac OSX 10.4. I tried it in both c++ and c with gcc 4.0.1 from xcode 2.3 and c++ onmandrivea 2006.0 w/ gcc 4.0.1-5mdk. The printf man page is not the easiest thing to read, so this could easily be a misinterpretation.

SEE ALSO: Not that the first solution in this thread is WROING (%su will print a string and a 'u'... that person probably meant to put in '%zu')... Google groups thread

// C++ version
#include <cstdio>
using namespace std;
int main (int argc, char *argv[]) {
  cout << "  sizeof size_t: " << sizeof(size_t) << endl;
  cout << "  sizeof    int: " << sizeof(int) << endl;
  cout << "  sizeof   long: " << sizeof(long) << endl;
  cout << "  sizeof  float: " << sizeof(float) << endl;
  cout << "  sizeof double: " << sizeof(double) << endl;
  cout << "  sizeof   char: " << sizeof(char) << endl;
  cout << "  sizeof  short: " << sizeof(short) << endl;
  cout << "  sizeof  void*: " << sizeof(void*) << endl;
  cout << "  sizeof long long: " << sizeof(long long) << endl;
  long long foo=1;
  printf ("printf pointer: %p\n", &foo);
  size_t bar=1;
  printf ("print size_t: %zu\n", bar);
  long biglong = __LONG_MAX__ - 2;
  printf ("printf __LONG_MAX__ -2: %ld",biglong);
  return 0;
}

Mac OSX makefile for 32/64 bit build/test

CPPFLAGS:=-Wall -g
foo:
	g++ ${CPPFLAGS} foo64.C -o foo64 -m64
	g++ ${CPPFLAGS} foo64.C -o foo32
	@echo
	@echo 64 bit
	@./foo64
	@echo
	@echo 32 bit
	@./foo32

And a straight C version of the minimal test:

/* C version */
#include <stdio.h>
#include <stdlib.h>
int main (int argc, char *argv[]) {
  long long foo=1;
  printf ("printf pointer: %p\n", &foo);
  size_t bar=1;
  printf ("print size_t: %zu\n", bar);
  long biglong = __LONG_MAX__ - 2;
  printf ("printf __LONG_MAX__ -2: %ld",biglong);
  return (EXIT_SUCCESS);
}

Here is the output of the Makefile. "-m64" is the key flag for gcc to make 64-bit binaries.

make
g++ -Wall -g foo64.C -o foo64 -m64
g++ -Wall -g foo64.C -o foo32
#
64 bit
  sizeof size_t: 8
  sizeof    int: 4
  sizeof   long: 8
  sizeof  float: 4
  sizeof double: 8
  sizeof   char: 1
  sizeof  short: 2
  sizeof  void*: 8
  sizeof long long: 8
printf pointer: 0x7fffeffffea60
print size_t: 1
printf __LONG_MAX__ -2: 9223372036854775805
32 bit
  sizeof size_t: 4
  sizeof    int: 4
  sizeof   long: 4
  sizeof  float: 4
  sizeof double: 8
  sizeof   char: 1
  sizeof  short: 2
  sizeof  void*: 4
  sizeof long long: 8
printf pointer: 0xbfffebc0
print size_t: 1
printf __LONG_MAX__ -2: 2147483645

http://www.wikimapia.org - Tag the world. I can't remember the system like this that I blogged about a few months ago. I like that this one is not just pushpins. You can define the region. Now if google could get better sat photos for some of the areas that I like.

There is a good chance that schwehr.org may be off for a couple weeks.

OpenGL Documentation Updates Now Available

I just tried it and right now XCode 2.3 can NOT build dpkg. So... my recommendation is to stay away from xcode 2.3 at least for a little while!

I did successfully build gmt 4.1.1, coin, and density using XCode 2.3 from the fink 10.4 tree and am looking forward to trying out the updated gdb debugger today.

From the Apple web site:

• New DWARF debugging format for improved debugging fidelity and efficient utilization of disk space
• New Distributed Network Build (DNB) scalable build architecture
• Numerous enhancements and bug fixes

My memory is failing me, but was it the STABS format 10 years ago that was the most common debugging format? If have been seeing a lot of Makefiles lately that compile with -ggdb for the GDB format. Hopefully, DWARF will help me debug my C++ issues that I have been facing the last few days.

Reading the GCC man page, it is unclear if -ggdb causes gcc to write DWARF already. How can I check the debugging format used inside of a binary?

http://developer.apple.com/tools/xcode/update.html. I guess I should go read about dsymutil.

Is there really a newer gdb in there or patches on the "current" gdb that I am getting:

GNU gdb 6.1-20040303 (Apple version gdb-434) (Wed Nov  2 17:28:16 GMT 2005)

Last week I tried to build a stock gdb and never succeeded. I wish apple would be quicker at getting code back into the trunks of the opensource software that they use for Mac OSX. A two year old gdb is giving me flashbacks to working on VxWorks where all of the tools that came with the os were so old, I just gave up and built the entire crosscompiler tool chain and debuggers myself.

From Version Tracker, it looks like xcode 2.3 is out. Version Tracker XCode 2.3 I think I will hold off a few days to see what others think before I attempt the 900+ MB download.

Roland pointed me at this C++ guide. It got us through the string type which I hadn't used in about 2 years. Strings are so much better than C style char * strings!

C++ Annotations

This is from the I can't believe this works department. I have been spending a while trying to track down a nasty crash when exiting from a large program. gdb seems to loose its ability to catch breakpoints part way through the exit process, so I am having trouble knowing where the crash really occurs or why. Last night I was wondering if it could be from mixed malloc/delete usage caused by strdup. The logical thing to do is to make a small test program that mixes the two and see if I can replicate the crash. I figured this would be pretty bad and hopefully have a similiar issue. It runs fine! I never expected that.

#include <cstring>
#include <iostream>
using namespace std;
static char *list[10000];



int main (int argc, char *argv) {
  for (int i=0;i<100;i+=1) {
    if (i%2==0) list[i]=strdup("foo");
    else list[i]=strdup("bar");
  }
  for (int i=0;i<100;i++) {
    cout << i << " " << list[i] << endl;
    if (i%2==0) free(list[i]);
    else delete [] list[i];
  }
}

USGODAE looks like an interesting source of data, but their bathymetry options do not look very useful.

I have never directly used graphviz before today. Yes, I run it all the time as a side effect of using doxygen, but that is not the same as creating my own graphs as dot files. Well, now I have and damn is it easy! The only thing to know is that the fink version for graphviz does not seem to be able to handle png images.

   Error: Could not find/open font : Times-Roman

I emailed the maintainer, but it does work with ps output. Here is my first example. I made a text file of a tree from within a GeoZui subtest:

TREE: I_am_root
        I_am_root -> c0; // 0
        I_am_root -> c1; // 1
                c1 -> cc0; // 0
                c1 -> cc1; // 1
        I_am_root -> c2; // 2
        I_am_root -> c3; // 3
        I_am_root -> c4; // 4
                c4 -> cd0; // 0
                c4 -> cd1; // 1
                c4 -> cd2; // 2

That works, but really is not that much fun. More fun is to use graphviz...

digraph foo {
        I_am_root -> c0; // 0
        I_am_root -> c1; // 1
        c1 -> cc0; // 0
        c1 -> cc1; // 1
        I_am_root -> c2; // 2
        I_am_root -> c3; // 3
        I_am_root -> c4; // 4
        c4 -> cd0; // 0
        c4 -> cd1; // 1
        c4 -> cd2; // 2
} // digraph foo

To view the graph either use the Mac GraphViz (which is pretty nice) or do this:

  dot -Tps -o foo.ps foo.dot
  gv foo.ps

And I ended up with a graph that looks like this:

Via Capt. Ben... http://www.assembla.com/

Distributed Software Teams



Start faster, and get more done, with less risk, and less
overhead. Add a software project and get free team building,
communication, trac, and subversion. Register as a developer or
service provider to get access to immediate paid trials. Post a Help
Wanted listing and discover unlimited resources. Launch your team with
our full-service team building and management.  Build your space



We offer public and private workspaces for individuals, teams, and
organizations of any size.  FREE Power Tools on Demand



Wiki * Issues * Files * Subversion * Trac * Typo * MORE

I didn't know Typo which is a blog system written in ruby 1.0.

SEE ALSO: A conversation with Andy Singleton about building global teams by John Udell.

As an aside, we just got some impressive thunder.

I was talking to someone yesterday who had not heard of this C/C++ trick, so I figured that is good enough reason to document it. I have no idea if I learned this from someone else or not.

The problem is one where you have a floating point numbers and there is not a clear case for floats or doubles. Floats are faster (at least on 32bit machines) and take less memory, whereas doubles take 2x the memory and a generally much slower. The trick is that you want to be able to switch easily and try out both. Maybe you will want to switch based on how fast the machine is or based on what kind of data you will be sending through the program. This is pretty easy to accomplish with a simple macro. You can make this much more complicated and allow for separate components to change independently. Here is the simplest case:

#define INTEGER int
//#define INTEGER short
#define REAL float
//#define REAL double
class locationUTM {
public:
      REAL easting;
      REAL northing;
      INTEGER zone;
};

Not the cleverest example, but by changing the definitions of INTEGER and REAL, you can quickly change all of the types that follow in that source file. Now it is up to you to make it snazzier with bells and whistles.

SeaTow's Boston Harbor Cam found via Panbo's blog.

UPDATE: read that wrong... that would be "Google Sketchup plugin released" which is for the Pro version.

Word on the street is that Google SketchUp is out for the Mac. Well, I am not seeing it on the free page... http://sketchup.google.com/download.html. It still says the mac version is coming soon. ARG.

That's it, no more gcc 3.3 for me. This business of mixing 3.3 and 4.0.1 is just not a good thing. This is the page that I am working from RangerRick's blog (his real name escapes me at the moment as I am sure happens to the other developers about me at times). BTW, his blog is full of lots of other good bits for information.

http://ranger.befunk.com/blog/archives/000654.html

NOTE: There are some recent changes to this proceedure. The fink cvs repository has moved (thanks to major troubles at sf.net). Here are the new commands.

  $ cd && mkdir foo && cd foo
  $ cvs -d :pserver:anonymous@fink.cvs.sourceforge.net:/cvsroot/fink login
  CVS password: 
  $ cvs -d  :pserver:anonymous@fink.cvs.sourceforge.net:/cvsroot/fink co -P -r branch_0_24 fink
  $ cd fink
  $ ./bootstrap.sh



  # Change cvs.sourceforge.net to fink.cvs.sourceforge.net in these two files:
  $ sudo emacs /sw/lib/fink/URL/cvs-repository
  $ sudo emacs /sw/lib/perl5/Fink/SelfUpdate.pm
  $ fink selfupdate-rsync  



  # Change the Trees to include unstable like this...
  # Trees: local/main stable/main stable/crypto unstable/main unstable/crypto
  $ sudo emacs /sw/etc/fink.conf

Only developers should use the cvs tree. Rsync is WAY faster.

Just what we do not need... more rain.

Roland showed me a nice way to create large strings (e.g. an html page) in C++ without getting into the hell that can be C style strings. Here is a minimal example:

#include <sstream>
#include <iostream>
using namespace std;
//
int main(void) {
  stringstream s;
  s << "foo" <<endl;
  s << "bar" << 2 << endl;
  cout << s.str() ;
  return 0;
}

Then building and running the demo:

$ cd && make ss CPPFLAGS="-g -Wall"
g++  -g -Wall   ss.C   -o ss
$ ./ss
foo
bar2

Note that s.str() returns a string object and to get a C string (const char *), use s.str().c_str().

I am finally back up-to-speed with the new fink cvs tree. I am pretty frustrated with sourceforge right now. I don't get a lot of time to work on fink, so any lost time has a big impact on what I can contribute. I hope fink moves to svn. Here is the trick that I got from the fink mailing lists that got me back to work:

cd /sw
find . -name CVS -type d \
-execdir perl -pi -e \
's|\@cvs.sourceforge|\@fink.cvs.sourceforge|' CVS/Root \; \
-execdir perl -pi -e 's|/cvsroot/fink/||' CVS/Repository \;

Looks like I will soon be moving the last of my macs to the fink 10.4 tree. On my server, I do have a copy of the 10.4-transitional tree, but I have been having problems with my work projects with gcc 3.3/gcc 4.x skew. It is just easier to go straight gcc 4.0.

http://www.cs.cmu.edu/~benhdj/Mac/unix.html - "UNIX Tips for Mac OS X: Here is a list of short tips on using various UNIX tools under Mac OS X - some of them might just be reminders for myself. Some of them are also applicable to other flavors of UNIX"

Videos by the author of PyDev for Eclipse: http://showmedo.com/videoListPage?listKey=PyDevEclipseList

Sony revives mini PC line for UMPC era looks interesting. Would be nice it if had a USB port, built in GPS, and cost a lot less ($1.3K). Then it would be an awesome device for ship interfaces. This would make a great AIS interface.

I have been exploring the space of C++ unit testing systems. I thought that CppUnit would be the way to go hands down. However, my first look into into CppUnit got me kind of frustrated with CppUnit. There was just so much typing to get anything done. After python, I am just not into that any more. I did some digging and finally found an interesting article that I liked: Exploring the C++ Unit Testing Framework Jungle. His conclusion is that CxxTest was the winner. However, his article is from Dec 2004. That is a long time in the software world. Some of the other test frameworks that look vaguely interesting:

• unit-- http://unit--.sourceforge.net/
• CUT http://sourceforge.net/projects/cut
• cpptest http://cpptest.sourceforge.net/
• http://freshmeat.net/projects/cxxtester/
• http://yaktest.sourceforge.net/
• cutee http://codesink.org/cutee_unit_testing.html
• TUT http://tut-framework.sourceforge.net/

The author of that report just released UnitTest++ http://unittest-cpp.sourceforge.net/ two years later. It looks that UnitTest++ is too new to be used in a big project yet. Bummer, as all the other projects looks like they have stagnated for two years.

Mac OS X for Oceanographers and Atmospheric Scientists (or anybody else who's used to Unix)

CppUnit looks like an interesting test framework, but unfortunately, the cookbook example is very hard to follow. Thankfully, I found the DirtSimpleCppUnitExample which does seem to work. This is yet another example of why I prefer to program in python. I can basically see what is going on, but it is way more complicated that why I have to do in python.

#include <memory>
#include <cppunit/extensions/TestFactoryRegistry.h>
#include <cppunit/ui/text/TestRunner.h>
#include <cppunit/CompilerOutputter.h>
#include <cppunit/TestCase.h>
#include <cppunit/extensions/HelperMacros.h>



class ExampleTestCase : public CppUnit::TestCase
{
  CPPUNIT_TEST_SUITE( ExampleTestCase );
  CPPUNIT_TEST( example );
  CPPUNIT_TEST( anotherExample );
  CPPUNIT_TEST( testEquals );
  CPPUNIT_TEST_SUITE_END();
  double	m_value1;
  double	m_value2;
  void	example ();
  void	anotherExample ();
  void	testEquals ();
public:
  void	setUp ();
};



CPPUNIT_TEST_SUITE_NAMED_REGISTRATION( ExampleTestCase, "ExampleTestCase" );



void ExampleTestCase::setUp ()
{
  m_value1 = 2.0;
  m_value2 = 3.0;
}



void ExampleTestCase::example ()
{
  CPPUNIT_ASSERT (1 == 1);
}



void ExampleTestCase::anotherExample ()
{
  CPPUNIT_ASSERT (2 == 2);
}



void ExampleTestCase::testEquals ()
{
  std::auto_ptr<long>	l1 (new long (12));
  std::auto_ptr<long>	l2 (new long (12));



  CPPUNIT_ASSERT_DOUBLES_EQUAL (m_value1, 2.0, 0.01);
  CPPUNIT_ASSERT_DOUBLES_EQUAL (m_value2, 3.0, 0.01);
  CPPUNIT_ASSERT_EQUAL (12, 12);
  CPPUNIT_ASSERT_EQUAL (12L, 12L);
  CPPUNIT_ASSERT_EQUAL (*l1, *l2);



  CPPUNIT_ASSERT(12L == 12L);
  CPPUNIT_ASSERT_DOUBLES_EQUAL (12.0, 11.99, 0.5);
}



CppUnit::Test *suite()
{
  CppUnit::TestFactoryRegistry &registry =
    CppUnit::TestFactoryRegistry::getRegistry();



  registry.registerFactory( &CppUnit::TestFactoryRegistry::getRegistry( "ExampleTestCase" ) );
  return registry.makeTest();
}






int main( int argc, char* argv[] )
{
  // if command line contains "-selftest" then this is the post build check
  // => the output must be in the compiler error format.
  bool selfTest = (argc > 1)  &&
    (std::string("-selftest") == argv[1]);






  CppUnit::TextUi::TestRunner runner;
  runner.addTest( suite() );   // Add the top suite to the test runner



  if ( selfTest )
    { // Change the default outputter to a compiler error format outputter
      // The test runner owns the new outputter.
      runner.setOutputter( CppUnit::CompilerOutputter::defaultOutputter(
									&runner.result(),
									std::cerr ) );
    }



  // Run the test.
  bool wasSucessful = runner.run( "" );



  // Return error code 1 if any tests failed.
  return wasSucessful ? 0 : 1;
}

The GNU makefile to go with it for mac osx with fink 10.4-transitional. The c++ compiler must match what was used to build CppUnit, so if you are using the 10.4 fink tree, chage g++-3.3 to just g++.

CXX:=g++-3.3
CXXFLAGS:=-I/sw/include 
LDFLAGS:=-lcppunit -L/sw/lib



dirtsimple:

Then build and run it:

$ make
g++-3.3 -I/sw/include   -lcppunit -L/sw/lib  dirtsimple.C   -o dirtsimple



$ ./dirtsimple 
...



OK (3 tests)

Okay. I think we have had enough rain now. Time to stop. People are pumping their basements, roads are being washed away, New Market was evacuated, and we have leaks in the walls and ceiling. Enough is enough.

Folks at CCOM have been worrying about time syncronization for multibeam and sidescan operations. The slashdot article, Computer Network Time Synchronization is very "timely" but has no discussion of IEEE 1588 (except for in one comment) which is disappointing.

It has been more than 10 years since I read the OpenGL red book. Here is a refresher:

Drawing Primitives in OpenGL which is a chapter from OpenGL Distilled.

http://www.tllts.org/ - The Linux Tech Show is a podcast that might be a substitute for now defuct "The Linux Show". I haven't listened in yet.

This looks like a pretty cool graphing program: vvidget. I haven't explored it much, but sounds great. It is a little scary that the install requires a reboot.

This entry in the Python Cookbook sight reminds me that I should really look into XML RPC. Logging SimpleXMLRPCServer by John Ferrell.

If you want to try out trac using a Mac OSX system with fink installed, it is pretty easy. Here are detailed instructions on the wiki (TracOnOsx, but this should work too.

  fink install svn-ssl svn-client-ssl
  fink install pysqlite2-py24
  fink install trac-py24
  mkdir ~/trac-foo && cd ~/trac-foo
  svnadmin create $HOME/trac-foo/svn
  trac-admin $HOME/trac-foo/myproj initenv
    Project Name> myproj
    Database connection string>  # just hit enter
    Path to repository> /Users/schwehr/trac-foo/svn # Replace /Users/schwehr!
    Templates directory> # just hit enter
  tracd --port 9000 $HOME/trac-foo/myproj

Now in a new terminal/xterm (or in your browser):

  open http://localhost:9000/

Modeling Object-Relational Bridge for python

  The Modeling framework intends to fill the gap between the python
  object world and relational databases. It relies on a model, based
  on Entity-Relationship Modelling, that describes how the two worlds
  map to each other. From your design of such a model, the database's
  schema and corresponding python classes are automatically
  generated. Thus, once you have designed how your classes should be
  stored in the RDBMS, you can focus on the real challenges - the
  logic of your business objects - while remaining in the
  object-oriented world of those objects and never having to worry
  about the SQL and RDBMS persistence layer below.

This was mentioned on the trac database backend web page along with SQLobject and SQLAlchemy. Haven't tried this one either.

ET is for ElementTree. Ok? Here is an improved print function for element trees that includes printing the tails. Check it out.

#!/usr/bin/env python
#from elementtree.ElementTree import parse # Element Tree
from lxml.etree import parse # lxml
#
def printET(ele,indent='  '):
    if None==ele: return 0
    if ele.keys():
	for name,val in ele.items():
	    print indent+ 'attr: '+name+' ... '+val
    c = ele.getchildren()
    count = 0
    for child in c:
	count += 1
	print indent+'tag -- '+child.tag
	if child.text and len(child.text.strip())>0: print indent+' text -- "'+child.text+'"'
	else: print 'WHAT?'
	num = printET(child,indent+'  ')
	if num > 0:
	    print indent+'end -- '+child.tag+' numchildren = '+str(num)
	if child.tail:
	    if len(child.tail.strip())>0:
		print indent+' tail -- "'+child.tail+'"'
    return count
#
root = parse('xml.xml').getroot()
printET(root)

The strip tries to get rid of white space only tails. The results:

  tag -- opening
   text -- "Dear John"
  tag -- body
   text -- "call Jane at "
    tag -- number
     text -- "603-555-1234"
     tail -- ".
Hurry, John. Time is running out!"
  end -- body numchildren = 1

The funny business with the tail comes from there being a newline embedded within the text before the end of the tag. Now I can get to all of the content in an xml file.

In my previous post, I wondered how to get the text within a tag that comes after a child tag. Looking through the lxml test suite, I found test_tail in test_elementtree.py that answered my question.

    def test_tail(self):
        ElementTree = self.etree.ElementTree
        f = StringIO('<doc>This is <i>mixed</i> content.</doc>')
        doc = ElementTree(file=f)
        root = doc.getroot()
        self.assertEquals(1, len(root))
        self.assertEquals('This is ', root.text)
        self.assertEquals(None, root.tail)
        self.assertEquals('mixed', root[0].text)
        self.assertEquals(' content.', root[0].tail)

Or here is what I did in ipython:

ipython
In [1]: from StringIO import StringIO
In [2]: f = StringIO('<doc>This is <i>mixed</i> content.</doc>')
In [3]: from lxml.etree import ElementTree
In [4]: root = ElementTree(file=f).getroot()
In [6]: print root.text
This is 
In [7]: print len(root)      
1
In [8]: print root[0]  
<Element i at 1178378>
In [9]: print root[0].text
mixed
In [10]: print root[0].tail
 content.

The key is that the text is the "tail of the child." Not exactly what I expected, but it works.

Here is a little puzzler. I was playing with an example from "Learning XML" and ran into a spot of trouble. The example was supposed to teach how to use entities to allow referencing them later in an xml file, but I hit a snag that is shown below. First the xml from the book but stripped down:

<?xml version="1.0"?>
<!DOCTYPE message SYSTEM "/xmlstuff/dtds/message.dtd"
[
<!ENTITY client "John">
<!ENTITY agent "Jane">
<!ENTITY phone "<number>603-555-1234</number>">
]>
<message>
<opening>Dear &client;</opening>
<body>call &agent; at &phone;.
Hurry, &client;. Time is running out!</body>
</message>

Now for the lxml python code to read it using the ElementTree API using a recurive desent algorithm:

#!/usr/bin/env python
#from elementtree.ElementTree import parse # Element Tree
from lxml.etree import parse # lxml
#
def printET(ele,indent='  '):
    if None==ele: return 0
    if ele.keys():
	for name,val in ele.items():
	    print indent+ 'attr: '+name+' ... '+val
    c = ele.getchildren()
    count = 0
    for child in c:
	count += 1
	print indent+child.tag,
	if child.text and len(child.text.strip())>0: print ' -- "'+child.text+'"'
	else: print
	num = printET(child,indent+'  ')
	if num > 0:
	    print indent+'end '+child.tag+' numchildren = '+str(num)
    return count
#
root = parse('xml-sample.xml').getroot()
printET(root)

Now running the python program, I end up with nothing after the inline number tag. How do I get at the trailing text? Clearly the text text is there as can be seen with this little python program:

#!/usr/bin/env python
import lxml.etree
root = lxml.etree.parse('xml-sample.xml').getroot()
print lxml.etree.tostring(root)

This 4 liner prints:

<message>
<opening>Dear John</opening>
<body>call Jane at <number>603-555-1234</number>.
Hurry, John. Time is running out!</body>
</message>

Boston Harbor simulator, and blowing stuff up [panbo's blog]

  This headline got my attention in yesterday's Boston Herald: "Tanker
  may be blown up to test Harbor worst-case scenario." Besides a
  general boyish interest in things blowing up, this story of course
  involves the LNG tankers that motor right alongside downtown Boston
  (and my temporary apartment) en route to their terminal.
...

Bergesen's pre-operational training of deck officers on new LNG carriers

  Bergesen regularly uses the Ship Manoeuvring Simulator Centre (SMSC)
  in Trondheim, Norway, for officers' conferences. In 2002 the company
  requested SMSC to develop a brand-new training programme that would
  focus on taking their new LNG carrier "Berge Boston" in and out of
  the Port of Boston, prior to its first port visit.
...

Simulator based training

  For many years MARINTEK has co-operated with Statoil, Navion and the
  Ship Manoeuvring Simulator Centre in Trondheim in the development of
  a simulator-based training programme for deck officers on
  offshore-loading tankers. Today our focus is on simulator-based
  training needs for the new generation of LNG carriers operating at
  all kinds of terminals including offshore loading.



  Simulators will be used for training of all deck officers on new LNG
  carriers as well as for developing and verifying tug boat assistance
  and operational procedures. Simulator training is a minor insurance
  cost for safer and more efficient operation of LNG carriers when
  approaching/leaving a terminal, whether it is land-based, fixed or
  floating.
...

This stuff looks very similar to the NASA Ames air traffic control simulator.

6.A.1. D11 - DISTRICT ELEVEN ATTENDED THE APRIL PFMC MEETING IN
       SACRAMENTO, CA FROM THE 3RD - 7TH. AT THIS MEETING THE
       FEDERAL COMMERCIAL AND RECREATIONAL SALMON FISHERIES WERE
       REOPENED (REF D). THE OTHER MAJOR ISSUE PRESENTED AT THE
       PFMC WAS HEARTACHE OVER THE IMPENDING AIS REQUIREMENTS. AIS
       WAS MANDATED THROUGH THE MARITIME TRANSPORTATION SECURITY
       ACT OF 2002 WITH A FINAL DEADLINE OF DECEMBER 31ST, 2004.
       THIS REQUIREMENT WAS DEFERRED, NOT EXEMPTED, FOR COMMERCIAL
       FISHING VESSELS CURRENTLY WITH VMS, WHILE A POSSIBLE
       SOLUTION FOR INTEGRATION BETWEEN THE TWO SYSTEMS COULD BE
       MADE. CURRENTLY, THERE IS NO FEASIBLE WAY TO INTEGRATE BOTH
       SYSTEMS AND AIS WILL NO LONGER BE DEFERRED FOR THESE
       VESSELS. THE CURRENT REGULATIONS ALLOTTED 14 MONTHS FOR
       IMPLEMENTATION OF AIS, SO THIS REQUIREMENT WILL NOT BE SEEN
       COMPLETE FOR AT LEAST THE NEXT YEAR AND HALF TO TWO YEARS.

I have written about the Python Testing Tools Taxonomy page before, but it is worth mentioning again and again.

What that? You say that you are not using any testing tools beyond print statements? Here are things that you should consider learning and the order that I suggest. WARNING: Entering preachy mode...

1. assert - Assert requirements throughout your code. For example, if a number must be greater than 0, say so!
2. pychecker and pylint definitely help to find bugs. Use them.
3. ipython - try while you code and use the ? for help
4. pdb - the python debugger. I still have trouble with emacs integration, but it is a good tool.
5. doctest - put testing in your documentation. Then users will know that the examples actually work.
6. unittest - they really are not hard to write if your code is designed right. Not all types of programs work with unittesting well (at least for me), but they are very powerful
7. Consider adding a "webserver backdoor". Writing a webserver in python is so easy that it is ridiculous (if you have a good example.) It is an easy way to expose your data structures for long running programs. Both Roland and I do this for our systems.
8. If you use xml in your python project, use xmllint and/or xmlstarlet to validate your xml!

I would like to look into twill and webunit soon, but they have not made it high up my priority list.

http://www.oilposter.org/. I haven't seen the details (the online font is too small), but it looks interesting.

Microscoft is now talking about SensorWebs. Microsoft's Plan to Map the World in Real Time:

  Researchers are working on a system that allows sensors to track
  information and create up-to-date, searchable online maps.

Not the most rigorous article as they are missing some important things like JPL's SensorWeb project and the fact that the maritime industry is rolling out a monster of a sensorweb... AIS.

To hold people over until sketchup becomes available for Mac OSX: http://www.sketchup.com/?sid=40

3D Warehouse

http://www.rmunn.com/sqlalchemy-tutorial/tutorial.html - SQLAlchemy tutorial... SQLAlchemy is a python database API that wraps sqlite, postgres, and oracle.

Here is another interesting binary parser. Unfortuntely, it does not appear to handle bit packings that have no relationship to being byte aligned (e.g. Maritime AIS). Still, this looks very useful for the majority of the binary formats out there.

http://hachoir.python-hosting.com/wiki/WriteYourParser

Here is an interesting example from where I work showing how these online maps don't totally get it right. There is no way on the interface to say, "Hey guys, it is really over here." Still the air photo is very nice to have and CCOM really needs to write something on the roof.

This came across the Coin mailing list this morning:

Dear all,



There is the second release of Open Inventor Tools on the 
http://merlin.fit.vutbr.cz/wiki/index.php?title=Open_Inventor_Tools



Features:
- new utilities: ivgraph and ivToInclude added
- ivperf meassuring algorithms overhaul
- textures are not loaded for cmd-line utilities (speed-up)
- other bugfixes/improvements (listed at http://merlin.fit.vutbr.cz/wiki/index.php?title=Open_Inventor_Tools_Issues)



Please, report bugs to my email.
Enjoy,
John

This reminds me that I need to finish my conversion of segy-py to seismic-py so I can get the paper submitted to a journal. I want to have it more as a standard python package before getting a paper out.

http://www.gcn.com/print/25_11/40660-1.html

House bill calls for $2.4 billion in grants for high-risk locations



Port security legislation approved by the House Homeland Security
Committee last month authorizes the award of $2.4 billion in grants to
high-risk ports over six years.



The grant money can pay for purchasing and upgrading security
equipment, including IT products, to enhance terrorism preparedness.



Eligible equip- ment is expected to include video surveillance
systems, waterside surveillance vessels, vehicle inspection stations,
closed-circuit telecommunications and access-control equipment for the
main gate.
...

Just found out that there is a new version of VLC out. VLC is a cross platform video player that plays some things that Apple's Quicktime will not play.

http://www.videolan.org/

I am getting into a large sized C++ application and am having some mac port issues. As a part of this, I would like to know how to demangle a c++ symbol name. The short answer:

echo "__ZN9Sv_Raster7GetSizeEPKcRiS2_12SVR_FileType" | c++filt 



  Sv_Raster::GetSize(char const*, int&, int&, SVR_FileType)

Don't know what I am talking about? When compiling C++ code, the complier generates a new name for each function that encodes the argument types. This lets the compiler handle operator overloading so that you can have multiple functions with the same name, but different arguments. For example, a sqrt function could have a faster version for floats compaired to doubles.

	float sqrt(float);
	double sqrt(double);

The next question is probably, "How are you seeing these mangled names?" That come from the compiled objects for each source file, which is the output from 'g++ -g -c foo.C'. That compile command will generate a "foo.o". To get the symbols, run "nm foo.o" and then read the man page for nm. 'T' is a symbol definition and a 'U' is where a symbol has been called and is "Undefined".

NOAA's s57 charts are available on the web here:

  http://ocsdata.ncd.noaa.gov/ChartServerV2.0/jsp/index.jsp

And instructions on working with them in mapserver are here:

  http://mapserver.gis.umn.edu/docs/reference/vector_data/S57

Now to do something with them!

[This is pretty much straight from the book Web Mapping Illustrated, but with a fink installed mapserver]

Yesterday, I made my first mapserver map. Just a command line thing, but it is a start. First I grabbed the 0.1 simple version of the world from the web:

  http://ftp.intevation.de/freegis/worlddata/

Not that is will probably complain on untar, but it still works. Here is my global.map:

MAP
  SIZE 600 300
  EXTENT -180 -90 180 90
  LAYER
        NAME countries
        TYPE POLYGON
        STATUS DEFAULT
        DATA countries_simpl
        CLASS
                STYLE
                        OUTLINECOLOR 100 100 100
                END
        END
  END
END

I then build a map with the command line mapserv. I will try to work on the info file so that mapserv is in the path, but it may be a few days.

  /sw/share/mapserver/shp2img -m global.map -o first-mapserver-map.png
  display first-mapserver-map.png

Today I played some with threads and sockets. Here is a very simple thread program. Here is a simple alice and bob type thread program. Talk about useless, but it does illustrate threads in a pretty simple way. This ends up with 3 threads.

#!/usr/bin/env python
import thread, time
def a():
    i=0
    while 1:
	i += 1
	time.sleep(.5)
	print 'a',i
def b():
    i=0
    while 1:
	i += 1
	time.sleep(.5)
	print 'b',i
##############################
thread.start_new_thread(a,())
thread.start_new_thread(b,())
for i in range(100):
    time.sleep(1)
    print i

I got an email from Jerry Isdale today about Delta3D and pyro. It has been more than a year since I posted a blog entry about the python robotics and I still have not yet checked it out. My excuse is that I was trying to finish my thesis. Now I have no more excuses.

http://www.Delta3d.org and http://www.pyrorobotics.org/. From Jerry:

  I've been connecting the two for UAVs, but UUVs wouldnt be that much
  of a stretch. The Delta3d stuff is from Naval Postgrad School in
  Monterey CA.  They've done a fair bit with underwater (and surface)
  vehicles in the past.

Looks like it is time to checkout what is going on over at NPS. It has been a long time since I have talked with anyone over there.

Shed Skin:

Shed Skin is an experimental Python-to-C++ compiler. It can convert
pure, but implicitly statically typed Python programs into optimized
C++ code. Currently, these programs cannot freely use the Python
standard library. 

A Glance at PyDev

Not a lot of detail, but anything is good. http://pydev.sourceforge.net/

Alpha release of NVRL for Java

...
NVDL is not so complimentary to Schematron; Schematron already allows
multiple namespaces fine. But NDVL will provide a great mechanism for
allowing your to selectively dispatch different parts of your document
to different validators. So you can pick the best schema language for
the job. Or, as is more often the case, you may be working with
different vocabularies each defined in a different schema language
(DTD, RELAX NG, XSD, Schematron, etc). 

Here is an interesting commercial software program. SeaTTY It is windows only and cost $33, but it is an interesting concept and it looks like it is very similiar to GNU radio.

  A program to receive weather reports, navigational warnings and
  weather charts transmitted in RTTY, NAVTEX and HF-FAX (WEFAX) modes
  on longwave and shortwave bands. Software also can automatically
  save NOAA Weather Radio SAME voice messages (NWR SAME) and them
  digital headers. No additional hardware is required - you need only
  a receiver and computer with a sound card.

NASA's XML registries page. This will send you off to the DOD page which requires a government sponsor to get access. Hmm. I can see these pages, but many others who might benefit cannot. I had a very hard time using the http://metadata.dod.mil/ site. Tons of stuff, but nothing that I would care about.

I want to be able to have a flexible lookup table system. As a part of that I don't know ahead of time what kind of information will be stored. This is an excuse to look at python "keyword arguments". This could be done with a dictionary, but I wanted to try it the python way.

#!/usr/bin/env python



def testkeys(foo,**keywords):
    print 'foo',foo
    print 'keywords',keywords



testkeys(1,blah=3,bar=5)



def testargs(first, *others):
    print 'first',first
    print 'others',others



testargs('1st',2,3,4)

Which when run...

./foo3.py



foo 1
keywords {'blah': 3, 'bar': 5}



first 1st
others (2, 3, 4)

I am not sure if XQuery is a useful technology for the stuff that I work on. Here is a tutorial on XQuery:

Learn XQuery in 10 Minutes

http://www.seacoastonline.com/news/05012006/news/100499.htm

...
UNH research associate professor and atmospheric chemist Jack Dibb and
research project engineer Eric Scheuer are on the jetliner among a
select group of scientists taking part in NASA's Intercontinental
Chemical Transport Experiment, or INTEX-B. This two-phase experiment
is aimed at understanding the transport and transformation of gases
and aerosols on transcontinental and intercontinental scales and
assessing their impact on global air quality and climate.

Is this SOFIA? Nope, Sofia is a Boeing 747SP. It must be the NASA DC-8 Airborne Laboratory

...
The DC-8 participated in the 2001 Transport and Chemical Evolution
over the Pacific (TRACE-P) mission. The study was one of a series of
NASA's Global Tropospheric Experiment aircraft missions aimed at
better understanding the human impacts on the chemistry of the global
troposphere (the atmosphere from the earth's surface to an altitude of
seven miles). The experiment began in the 1980s as a tropospheric
chemistry experiment. Today it is providing a powerful new approach to
understanding the profound changes human activity generates and the
possible long-term effects these changes may have on the habitability
of Earth.
...

I met an interesting person on my flight from Baltimore to Manchester, NH last week. Hamish runs tours down to Antarctica and other exotic locations on his sailboat. The Seal has a lifting keel and ruder.

http://expeditionsail.com/

Here is a quick graph of compile times of Coin 2.4.5 on a Quad G5 Mac with increasing levels of make parallization (using e.g. make -j 4). Compile time is in minutes.



This shows that after "make -j 3" there is not much improvement in compile time. It also shows that there is a slight improvement for going beyond the number of CPUs. Note that this timing included the configure and install phases which are not parallelized. Therefore this contains a substantial constant time offset.

I want to be able to use a strings as file like objects for testing objects that work on files, Turns out there is exactly a class designed for this. Normally reading a file can be done like this:

#!/usr/bin/env python
for line in open('datafile'):
    print 'line',line,

The StringIO class has the requisite interface so that it can work the same way.

#!/usr/bin/env python
s='''first
second
'''
from StringIO import StringIO
for line in StringIO(s):
    print 'line',line,

When run, this gives:

line: first
line: second