11.30.2009 12:09

AIS on the International Space Station

Columbus to host worldwide sea traffic tracking experiment 
ESA's Columbus module on the International Space Station is being
fitted with experimental hardware to track the passage of ships across
the world's oceans. A pair of European-built signal receivers designed
for wide-area vessel detection are being ferried to the orbital
outpost aboard Japan's newly-launched H-II Transfer Vehicle.
...
Not the most cost effective way to deploy AIS, but amusing.

Posted by Kurt | Permalink

11.29.2009 18:13

hdf in fink - success and failure

Several of us have been looking at trying to get hdf 4 to work properly in fink. If you know how to make it work on 10.5 PPC/Intel and 10.6, help would be greatly appreciated. HDF 4 has been a problem child for a while. The initial failure of HDF trying to build itself with the xlf compile (never heard of this one before) was fixed with a F77=gfortran configure option, but then it fails in C code with uint16 not being defined. Very strange. The details are here: http://paste.lisp.org/display/91241#1. It would be nice to have a single info file for hdf4 again that does the right thing. Thanks to Raymond Garcia for taking the lead on getting HDF 4 to work on 10.6. We also have not yet looked at hdf4 in 10.6 64-bit builds.

Posted by Kurt | Permalink

11.28.2009 18:48

Using QGIS to view PostGIS data

I've tried several times in the past to get my data from PostGIS to show up in QGIS and I never figured it out until today. The punch line is to create you database tables using "WITH OID". I had created two point and 1 line tables in my database. In the add PostGIS layers dialog, I saw my 3 tables, but there was a blank in the primary key column. The first problem is that the QT drop-down selector doesn't show. If you click in that blank space, you will see the possible key fields. This has nothing to do with what you call your PRIMARY KEY when you create the table.



The second problem I had was that the "Add" button was always grey. Thanks to a kind person on the QGIS IRC channel, I now know that I have to create my table with knowledge of the PostgreSQL OID information. OIDs are number assigned to each row in PostgreSQL that are unique across all tables and databases. For example, in my ais_pg_transitlines_noMakeLines.py script, here is my new create statement: 
CREATE TABLE tpath
(
  id INTEGER NOT NULL REFERENCES transit(id),
  userid INTEGER NOT NULL
) WITH OIDS;
SELECT AddGeometryColumn('tpath','track',4326,'LINESTRING',2);
I can now see the oids in my table: 
SELECT id,userid,oid FROM tpath LIMIT 10;
 id | userid |  oid  
====+========+======
  1 |      0 | 35963
  2 |      0 | 35965
  3 |      0 | 35967
  4 |      0 | 35968
  5 |      1 | 35970
  6 |      1 | 35971
  7 |    310 | 35973
  8 |    310 | 35975
  9 |  80396 | 35977
 10 |  80399 | 35979
And with that, I can now add my layer into PostGIS and view the lines:



Only trouble is that I'm getting this error: 
Your PostGIS installation has no GEOS support.
Feature selection and identification will not work properly.
Please install PostGIS with GEOS support (http://geos.refractions.net)
I do have GEOS installed in my PostGIS installation. Things like this work: 
SELECT ST_Extent(track) FROM tpath;
                                 st_extent                                 
==========================================================================
 BOX(-70.5970764160156 42.093505859375,-70.0352401733398 42.7661170959473)
But digging farther, I sliced and diced my PostGIS install: 
SELECT ST_Buffer(track,100) FROM tpath LIMIT 1;
ERROR:  buffer:: operation not implemented - compile PostGIS with JTS or GEOS support
The PostGIS PG/SQL code is loaded from this file: /sw32/share/doc/postgis83/lwpostgis.sql We can open it up and see what buffer is doing: 
CREATE OR REPLACE FUNCTION buffer(geometry,float8)
   RETURNS geometry
   AS '/sw32/lib/postgresql-8.3/liblwgeom.so','buffer'
   LANGUAGE 'C' IMMUTABLE STRICT; -- WITH (isstrict,iscachable);
Time to follow that file path to the shared library. 
ls -l /sw32/lib/postgresql-8.3/liblwgeom*
-rwxr-xr-x  1 root  admin  504388 Oct 25 21:19 /sw32/lib/postgresql-8.3/liblwgeom.1.3.so
lrwxr-xr-x  1 root  admin      16 Oct 25 21:19 /sw32/lib/postgresql-8.3/liblwgeom.1.so -> liblwgeom.1.3.so
lrwxr-xr-x  1 root  admin      16 Oct 25 21:19 /sw32/lib/postgresql-8.3/liblwgeom.so -> liblwgeom.1.3.so
Macs have otool, which will let us see what a library is linked against (similar to ldd on normal Unix machines). 
otool -L /sw32/lib/postgresql-8.3/liblwgeom.1.3.so
/sw32/lib/postgresql-8.3/liblwgeom.1.3.so:
        /sw32/lib/libproj.0.dylib (compatibility version 7.0.0, current version 7.6.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 124.1.1)
On one of my Mac OSX 10.5 machines, I can see that libgeos3 did indeed make it into the build. 
otool -L /sw/lib/postgresql-8.3/liblwgeom.1.3.so
/sw/lib/postgresql-8.3/liblwgeom.1.3.so:
        /sw/lib/libgeos3/libgeos_c.1.dylib (compatibility version 6.0.0, current version 6.2.0)
        /sw/lib/libproj.0.dylib (compatibility version 6.0.0, current version 6.5.0)
        /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.1.4)
Back to the drawing boards.

Update 29-Nov-2009: Recompiled postgis83 1.3.5 and it works now. I have no idea why it didn't get libgeos3 linked in correctly before.

Posted by Kurt | Permalink

11.25.2009 09:32

AGU gives Author Choice

Woohoo! This is good. However, I just checked how much one of my published papers would cost to release: $1338.75. Why am I using journals again? Oh yeah... permanent archive, peer review, and career credit. If being a professor didn't require Journal publications, I would definitely go elsewhere. Author Choice for Open Access 
AGU journals now offer authors the opportunity to make their articles
open for others to read for free. Authors choosing this option pay a
fee based on article length and number of figures; these charges are
designed to offset the potential loss of subscription income.
It's clear from looking at Web of Science, that very few of my true journal papers are being used. The impacts of my non-journal activities are much greater: giving talks, presenting posters, blogging, and posting on YouTube. Sigh.

Posted by Kurt | Permalink

11.24.2009 07:49

Icon heading in Google Earth - broken?

I throught I would give the >heading< tag in KML a try. At first, the results looked great:



When I rotate the camera, the ship position reports disintegrate. It really seams like the heading should apply relative to the earth, not relative to the camera! This is a bug in the Mac version. Google Earth on Windows XP appears to work correctly.



Try it for yourself: ship.kmz

Also, Google Earth definitely needs a more compact (both in terms of disk space and memory when loaded) representation of Points. I wish that Points could look something like this: 
<Placemark>
  <name>ship</name>
  <Point heading="315.2" icon="tug.png" color="ff0000ff" scale="0.5" unix_timestamp="1259066628.96">
    <coordinates>-70.39408,42.35226,0</coordinates>
  </Point>
</Placemark>
With other software that I use, it's not a big deal to pull in millions of points. Google earth starts to slow down with just 3K points.

Posted by Kurt | Permalink

11.21.2009 10:29

Updated diagram for the RTCM sector zone message

At the RTCM meeting last week, I created an updated illustration for how the sector zone type should work. There was some confusion before on how sectors worked with the previous figure that only showed the left hand case. Now the right hand case shows what we described in the text for a zone that spans true north.


Posted by Kurt | Permalink

11.21.2009 10:23

Inside the Healy

A quick and partial tour of the inside of the Healy.





This is the aft control area for the ship and the winches. This is just aft of the winch room on the starboard side.



Here is the Knudsen subbottom profiler and a backup port side. Just out of view is the rack for the gravity meters. The gravity meter rack rack component heaters must be kept warm or it will take 6 months to recalibrate them, so they have a couple days worth of battery backup.



Here are the actual gravity meters. Not to be used as a stable platform for sodas or work tools.



The rack for the old Seabeam 2100 that is being removed.


Posted by Kurt | Permalink

11.21.2009 09:44

Healy in dry dock

This week I visited Seattle for eNav09, the RTCM AIS for VTS working group, and some meetings with the Seattle NOAA folks.



I got a real treat on the last evening: Dale Chayes gave 3 of us a tour of the USCGC Healy Ice Breaker while she is in the Todd Shipyard for maintenance and to have the Seabeam multibeam replaced with a new and much more accurate multibeam sonar. I got a bunch of photos with my iPhone, so unfortunetly, they are pretty grainy in the low light / high contrast conditions that exist in the dry dock. I've been on a US Navy cutter in dry dock before, but I never got to get down under that vessel.

This first image is from up on the wall of the dry dock looking at the aft / starboard end of the ship. The white columns in the middle are dry dock supports to distrute the weight of the ship. You can see the starboard propellar shaft as the propeller and ruder have been removed.



This is right behind the ship. You can see the same white columns for support and both propeller shafts. From this view on, everything in the photos is stuff that is underwater and out of sight normally.



The port propeller shaft has been pulled out about 10 feed. There is a onstruction worker on the right for scale.



The ice knife is under the bow to help break up ice.



Here is one of the OSU guys holding up a 2x4 for scale of a dent that was discovered once the dry dock was raised out of the water. The crew does not know when this happened. Breaking ice is a noisy afair and the ice that did this likely didn't really cause much of a bump compared to the normal ice breaking.



This looking up the side of the ship so that you can see how deap the dent is. Again, this is a 2x4 for scale



The main reason we were out at the dry dock was to check out the upgrade of the multibeam. The old transmit array was up in the aft end of the ice knife. The array has been removed, but the old window will be put back in the ship incase some other instrument needs that space in the future. I wasn't able to get a picture in the very dark machining area, but the new window frame is being cut next next week. The steel frame will have a very large single titanium window.



This is directly under the center of the center of the ship. The crews are cutting through the fuel tanks area here to make room for the much larger transmit array.



This is the reworked larger area for the receive array. The receiver array's long direction runs athwart ship. This is the typical Mills Cross array layout.


Posted by Kurt | Permalink

11.17.2009 16:18

Legal aspects of AIS

I am currently listening to Alan Weigel's talk a eNav09. His talk is "Don't Forget the Legal Side". He just talked about his court case:

Cresent Towing & Salvage v. M/V CHIOS BEAUTY, Civ. No. 05-4207, 2008 U.S. Dist. LEXIS 62247 (E.D. La. Aug. 14, 2008): 
The Court does not find the AIS evidence to be conclusive evidence of
individual vessel movement.
I have to disagree with the court on this one. Sometimes you can say that the postitions are not accurate, but in most cases, they are accurate.

Posted by Kurt | Permalink

11.17.2009 09:17

New version of Aldebaran II whale notices

Sean put in some hard work and just in time for eNavigation 2009: we have a whale notice display that is working!




Posted by Kurt | Permalink

11.16.2009 10:53

mdbtools for working with MS Access databases

I'm working on some schema migrations for a MS Access database and again mdbtools is proving to be very helpful. I started off thinking about writing to tool to parse the XML XSD schema output, but while not to hard, it was going to take some time. 
grep '
<xsd:element name="key" minOccurs="1" od:jetType="autonumber" od:sqlSType="int" od:autoUnique="yes" od:nonNullable="yes" type="xsd:int">
<xsd:element name="ood" minOccurs="1" od:jetType="longinteger" od:sqlSType="int" od:nonNullable="yes" type="xsd:int">
<xsd:element name="rid" minOccurs="1" od:jetType="longinteger" od:sqlSType="int" od:nonNullable="yes" type="xsd:int">
<xsd:element name="sighting_date_time" minOccurs="1" od:jetType="datetime" od:sqlSType="datetime" od:nonNullable="yes" type="xsd:dateTime">
<xsd:element name="species" minOccurs="0" od:jetType="longinteger" od:sqlSType="int" type="xsd:int">
<xsd:element name="group_size" minOccurs="0" od:jetType="longinteger" od:sqlSType="int" type="xsd:int">
<xsd:element name="longitude" minOccurs="0" od:jetType="double" od:sqlSType="float" type="xsd:double">
<xsd:element name="latitude" minOccurs="0" od:jetType="double" od:sqlSType="float" type="xsd:double">
<xsd:element name="geographic_accuracy" minOccurs="1" od:jetType="longinteger" od:sqlSType="int" od:nonNullable="yes" type="xsd:int">
<xsd:element name="region" minOccurs="1" od:jetType="longinteger" od:sqlSType="int" od:nonNullable="yes" type="xsd:int">
<xsd:element name="species_id" minOccurs="1" od:jetType="longinteger" od:sqlSType="int" od:nonNullable="yes" type="xsd:int">
<xsd:element name="behav1" minOccurs="0" od:jetType="longinteger" od:sqlSType="int" type="xsd:int">
<xsd:element name="behav2" minOccurs="0" od:jetType="longinteger" od:sqlSType="int" type="xsd:int">
<xsd:element name="duplicate" minOccurs="0" od:jetType="longinteger" od:sqlSType="int" type="xsd:int">
...
Instead, I gave mdbtools a try... 
mdb-schema -T sightings new-schema.mdb
mdb-schema -T sightings sightings-prototype-v05.mdb 
CREATE TABLE sightings
 (
        sighting_date_time      DateTime (Short), 
        group_size              Long Integer, 
        longitude               Double, 
        ood                     Long Integer, 
        region                  Long Integer, 
        species_id              Long Integer, 
        duplicate               Long Integer, 
        reporting_source        Long Integer, 
        reporting_entity        Long Integer, 
        key                     Long Integer, 
        rid                     Long Integer, 
        behav1                  Long Integer, 
        behav2                  Long Integer, 
        action                  Long Integer, 
        latitude                Double, 
        comments_private        Text (500), 
        comments_public         Text (500), 
        species                 Long Integer, 
        geographic_accuracy     Long Integer
);
A little text replacement transformation makes this valid sql... 
CREATE TABLE sightings
 (
        sighting_date_time      TIMESTAMP, 
        group_size              INTEGER, 
        longitude               REAL, 
        ood                     INTEGER, 
        region                  INTEGER, 
        species_id              INTEGER, 
        duplicate               INTEGER, 
        reporting_source        INTEGER, 
        reporting_entity        INTEGER, 
        key                     INTEGER, 
        rid                     INTEGER, 
        behav1                  INTEGER, 
        behav2                  INTEGER, 
        action                  INTEGER, 
        latitude                REAL, 
        comments_private        VARCHAR (500), 
        comments_public         VARCHAR (500), 
        species                 INTEGER, 
        geographic_accuracy     INTEGER
);
This looses all the info about foreign and primary keys, constraints, and other important db information, but it gets me pretty close to what I need.

Posted by Kurt | Permalink

11.14.2009 19:24

State of PostGIS video

Paul Ramsey has a great video online: The State of PostGIS. This is a great overview talk. I am really excited about the Geography feature that should be out soon. It adds proper handling of the Earth as a sphere to PostGIS. This will be a huge improvement for global seafloor mapping projects. I am also watching the progress of the PostGIS in Action book. I will definitely be picking up a copy.




Posted by Kurt | Permalink

11.14.2009 13:06

Whale notice change

I've just changed the zone notice for no whales detected to expire after 20 minutes. It was 10 minutes before. If the there is substantial clock drift on the computer displaying the AIS messages, the no whales observed can flap... it can dissappear and reappear in a slow flicker. This shows how critical it is to have computers track time and that Windows Time Service is just not up to the task. Here is a sample of a decode of a message with the new delay time: 
zonemsg_decode.py -v '!AIVDM,1,1,,B,803OvriK`U07Aj0Vh3nt61A9DT;0,0*2D,d-090,S0285,t175107.00,T07.60366941,r00369945,1258221067'
{'DAC': 366, 'RepeatIndicator': 0, 'UserID': 3669739, 'Spare': 0, 'MessageID': 8, 'FI': 34}
ZoneMsg
  dac: 366
  fi: 34
  zone_id: 10
  zone_type: 0 [Right Whales NOT OBSERVED] 
  start_utc_day: 14
  start_utc_hour: 17
  start_utc_minute: 50
  zone_duration_minutes: 20
  num zones: 1
  ZoneCircular 9260m radius at (-69.85239, 42.08153)
  ZoneCircular
        radius:         9260m
        longitude:      -69.8523866667
        latitude:       42.0815283333
  Now:        2009-11-14 17:59:19.986791
  Start time: 2009-11-14 17:50:00
  Expires:    2009-11-14 18:10:00
  Remaining:   640 sec
  Remaining:   10.67 min
  Remaining:   0.18 hr
NOTE: this is the old style of zone from last summer. The final IMO message will differ slightly from this. I've heard that IMO is going to change things like the order of Longitude and Latitude to be different than in the ITU AIS specification.

Posted by Kurt | Permalink

11.12.2009 09:36

CCOM Wave Tank Maintenance

Andy and Paul have drained the wave tank for periodic maintenance... a view of the tank without any water in it.


Posted by Kurt | Permalink

11.10.2009 09:29

Whales and noise in Physics Today

Human-generated sound and marine mammals, Loud anthropogenic noises can alter the behavior of whales and other marine mammals, sometimes with fatal consequences by Peter Tyack.


Posted by Kurt | Permalink

11.09.2009 21:11

Port tunneling with socat, not netcat or ssh tunneling

Chaoyi pointed me at socat for network plumbing issues. I just gave it a go forwarding AIS data from one machine to another and it work great! On the gateway: 
socat TCP4-LISTEN:9999 TCP4:aissource:12345
Then, the outside host gets the data like this: 
socat - TCP4:gateway:9999
!AIVDM,1,1,,A,15MpsWgP2=Ji0APGRhsefOwP2L0`,0*4A,d-111,S1765,t015347.00,T47.08294235,r003669959,1257818027
!AIVDM,1,1,,B,15Mvw:0P43rjsePFl?J4;?wN0@Lp,0*74,d-090,S1773,t015347.00,T47.28274625,r003669959,1257818028
!AIVDM,1,1,,B,35PlLL0OhbJlcK:G7FS:1`GT20u0,0*04,d-096,S1876,t015350.00,T50.02859357,r01SSHI1,1257818030
!AIVDM,1,1,,B,16H;FR002lrn2DLG77oSL2kN0D0R,0*0B,d-091,S1879,t015350.00,T50.1100255,r01SSHI1,1257818030
!AIVDM,1,1,,B,403OwoiuVm1mjrdhThG>od700D2`,0*45,d-108,S1881,t015350.00,T50.16316332,r01SSHI1,1257818030
!AIVDM,1,1,,A,15Mvw:0P43rjsk6Fl??T;?wR08N0,0*2C,d-100,S1882,t015350.00,T50.20326732,r01SSHI1,1257818030
!AIVDM,1,1,,B,16K>Rf002JrpQs8G49edq9OP00SF,0*0A,d-085,S1776,t015347.00,T47.37663907,r003669959,1257818028
...
socat even has options to do ssl over the tunnel if I need to protect the stream. Nice! I thought netcat was powerful, but socat is even better. I could have also used something like proxy, but why not just learn socat and head towards all the power of socat?

Posted by Kurt | Permalink

11.09.2009 13:40

Finding bad points in AIS

We are working through the dataset trying to remove any trouble spots. I've made a kml for Google Earth that has all the points where we have 10 minutes or more between position reports and/or 1km distance. This gives us about 400 points from the 3.3 million that we need to carefully check. This analysis is made more challenging because of poor time control on the position reports (remember to use NTP and/or log GPS time with your data!)

Here I'm showing one suspect point that actually looks pretty good. It shows that the coverage is pretty spotty on the eastern side of the study area.



However, Brad has checked all of our 3000+ transits and discovered this troublesome track:



Unlike my previous trouble that turned out to be a bug in handling points in my code, I am pretty sure that this is trouble with GPS on the ship. Our first pass algorithm for trouble detection is definitely missing this kind of thing. Careful tracking of reported speed over ground (SOG), course over ground (COG), heading, and position derived values with time should be able to catch this type of thing, but I don't have a solid time basis for this data. Best guess is that this is multipath with recovery.


Posted by Kurt | Permalink

11.09.2009 10:40

Preventing updatedb from scanning attached drives

On my Mac, I don't want updatedb searching my Time Machine backup drive (or any other attached media), so I just tried tweaking the /sw/etc/updatedb.conf file to exclude some areas. I can't seem to do more than just one. If I add /private/tmp, I get this complaint: 
sudo /sw32/etc/cron.daily/findutils
/sw32/etc/updatedb.conf: line 11: /private/tmp: is a directory
But this config seems to mostly work okay... 
DAYS=1
STATUS=on
PRUNEFS=/Volumes
However, find seems to have some trouble with directories: 
/sw32/bin/find: /usr/share/man/man3: Cannot allocate memory
I find myself using both mdfind (macosx spotlight) and locate depending on what I am trying to track down.

Posted by Kurt | Permalink

11.08.2009 14:51

IMO AIS Discrepancy Reports

MarineLink just posted about IMO's - AIS Discrepancy Reports: MSC.6/Circ.4 AIS Discrepancy Reports - July to September 2009 (Quarterly report) (PDF). The report for those months has about 50 vessels that had broadcast incorrect information. I have to say that this list is not very helpful. It's reports only a small fraction of the what is going on there with errors. There isn't a single mention of a MMSI of 0, 1, 1193046, etc.

An example from just three receivers in the Boston area that were seeing the nauticast error / default state of a MMSI of 1193046. This is from 2008, but I am sure that 2009 is not much better, because there is no effort to push these vessels to fix things. 
echo 'select distinct(name) from shipdata where userid = 1193046;' | psql ais2008 | sort -n
 ABE LINCOLN
 ASPHALT SEMINOLE
 CAMERONS POINT
 ENTERPRISE
 LOUKAS I
 LUCINDA SMITH
 MV TRADER
 MY MUSE
 NAUTICAST
 PACIFIC GUARDIAN
 PI MANHATTAN
 WILLIAM BRECKINRIDGE
 ZIM PANAMA
A PDF circular is not a helpful publication for those in the community who are using this data. See Calder and Schwehr, 2009 for a more detailed view of trying to validate AIS.

Posted by Kurt | Permalink

11.05.2009 05:34

Collecting data on Google Android phones

This is a very cool project! Think environmental response, Coast Pilot updates, chart updates. If only it worked on the iPhone too. Thanks to Eric Park of the Disaster Cam / GeoCam team at NASA Ames for the pointer.

Open Data Kit (ODK)


Posted by Kurt | Permalink

11.04.2009 06:57

Fall is finished

The leaves are now off the trees in the yard and there is an enormous pile of leaves obscuring the compost pile.




Posted by Kurt | Permalink

11.03.2009 15:48

Speaking Weds and Thurs this week

I am giving two talks in NH this week. Tomorrow (Weds Nov 5), I will be at Colby-Sawyer College in New London, NH. 
Landing Robots on Another Planet

Ivey Science Center Room 201
    Colby-Sawyer College
      New London, NH

This presentation will give an insider's view of landing robots on
another planet. Research assistant professor has Kurt Schwehr Mars has
worked on NASA mission control teams and designed computer vision, 3D
visualization, and on-board driving software for NASA's Mars
exploration program.
And the next day, I'll be speaking at UNH: 
Visualizing Mars Exploration

CS900: Thursday, November 5th, 12:40-2:00
       N113, Kingsbury Hall
      Univ. of New Hampshire

With the successful landing of the Phoenix Mars Lander comes the task
of visualizing the spacecraft, its operations and surrounding
environment. It requires a diverse team and set of software to provide
a suite of visualizations that shed light on the operations of this
visitor to another world. The core set of tools range from web-based
production tracking (Image Products Release Website), to custom 3D
transformation software, through to studio quality 2D and 3D video
production. Ground data systems from bash to java support the
visualization team in their production of still and video content that
is delivered to the public. Recently the Google Earth "Mars Mode" has
enabled the casual user to experience missions at home in the spatial
context of the planet in a way that was only available on super
computers just a few years ago. This presentation focuses on the data
management and visualization aspects of recent NASA Mars surface
missions.

Posted by Kurt | Permalink

11.03.2009 10:12

Transit analysis Google Earth Viz

I found and squashed the that gave me jitter in the latitude... I was not properly splitting up the WTK "POINT(x y)". Here are three transits to show how we are currently looking at the transits in great detail. The color codes show which receiver recorded the position report. I take the first station to report in and remove the other 2 position reports as duplicates.

2008-transit-analysis.kmz


Posted by Kurt | Permalink

11.02.2009 15:48

X11 crash in 10.6

I'm using the Apple X11 in Mac OSX 10.6 and I'm not very happy with it. In addition to the Apple key only sort of working like the meta key (Apple-F moves forward a word and capitalized in a xterm bash), the whole xserver just crashed. Thanks. 
Nov  2 15:44:13 catbox [0x0-0x2d92d9].org.x.X11[6029]: X11.app: Launching /usr/X11/bin/xterm:
Nov  2 15:44:13 catbox [0x0-0x2d92d9].org.x.X11[6029]:  argv[0] = /bin/sh
Nov  2 15:44:13 catbox [0x0-0x2d92d9].org.x.X11[6029]:  argv[1] = -c
Nov  2 15:44:13 catbox [0x0-0x2d92d9].org.x.X11[6029]:  argv[2] = /usr/X11/bin/xterm
Nov  2 15:44:14 catbox org.x.startx[6136]: Xquartz: start_x11_server: (ipc/mig) server died
Nov  2 15:44:14 catbox com.apple.launchd.peruser.501[132] ([0x0-0x2d92d9].org.x.X11[6029]): Exited with exit code: 1
Nov  2 15:44:14 catbox org.x.startx[6136]: /usr/X11/bin/xinit:  connection to X server lost.
Nov  2 15:44:14 catbox [0x0-0x2d92d9].org.x.X11[0]: xterm:  fatal IO error 32 (Broken pipe) or KillClient on X server "/tmp/launch-CfHvNS/:0"
Nov  2 15:44:15 catbox login[6261]: USER_PROCESS: 6261 ttys000
Nov  2 15:44:16 catbox org.x.privileged_startx[6149]: font_cache: Done

Posted by Kurt | Permalink

11.02.2009 06:25

ssh tunneling made easier

At CCOM, we have a nat'ed address space for 99% of our systems. They live in a a ccom.nh local domain spread over a bunch of 192.168.*.* subnets. This reduces the exposure of our windows machines to trouble from the outside internet and lets our IT team manage IP addresses without having to interact with with university IP networking team most of the time. For me, this presents a challenge as we do not have VPN setup. Some CCOM members get to their Windows boxes via Tunnelier (not free software). I use Macs that come standard with OpenSSH. I have a number of bash alias that make life easier, e.g.: 
alias wikitunnel='ssh -L 8080:wiki.ccom.nh:80 ccom.unh.edu'
alias wiki='open http://localhost:8080'
alias mymactunnel='ssh -L 8001:mymac.ccom.nh:22 ccom.unh.edu'
alias mymac='ssh -p 8001 localhost'
The first two let me use the internal wiki website remotely, while the second two let me lot into my Mac desktop. Managing all those port numbers is no fun and it requires I keep my port numbers straight. Chaoyi recently pointed out that, for ssh, having all these aliases is not required. I now have a file that manages the proxy for ssh into CCOM for me: ~/.ssh/config: 
Host *.ccom.nh
ProxyCommand ssh USERNAME@ccom.unh.edu /home/USERNAME/bin/nc -w 1 %h %p
Change the USERNAME to be your login and tweak the home path to point to where ever nc is installed. I had to put a copy of netcat in my bin directory, but now I am able to connect to any machine inside our domain that has sshd available. 
ssh snares.ccom.nh
Last login: Mon Nov  2 06:25:42 2009 from gateway.ccom.nh

# schwehr@mymac.ccom.nh 1 $ cd && exit
logout
Connection to mymac.ccom.nh closed.
Killed by signal 1.
That killed by sig 1 is a bit weird, but I don't notice it any more. scp also works for copying files: 
scp mymac.ccom.nh:foo bar
foo                                       100% 5869     5.7KB/s   00:00    
Killed by signal 1.
Not as easy as living in a VPN. However, it works and what would you do if you had to be VPN'ed into 2 or 3 different institutions to get your job done?

Trackback: Val mentions using fuse and autossh for SSH tunneling [Val's Brain Log]

Posted by Kurt | Permalink

11.02.2009 06:24

Mastering django resources

Mastering Django - Part 1 [Ryan Kaskel] 
...
This article is intended to be a collection of resources for a Python
programmer who wants to learn Django. You don't need to be an expert,
but I assume that you have read an introductory book on Python and
know the basics. I hope this guide will prevent other programmers from
giving up on Django like I first did and help them master the web
application framework. ...

Posted by Kurt | Permalink