Kurt's Weblog

g-cubed is pretty speedy. Now waiting for the decision... talk about detail...

Stage                                   Start Date
=====                                   ==========
Waiting for Editor Decision             2005-11-10 10:01:17
Editor Assigned                         2005-11-10 10:01:17
Waiting for Editor Assignment           2005-11-10 10:01:17
Initial Quality Control Complete        2005-11-10 10:01:16
Initial Quality Control Started         2005-11-07 18:17:25
Author Approved Converted Files         2005-11-07 18:17:25
Waiting for Author Apprv of Conv Files  2005-11-07 18:00:09
File Conversion Complete                2005-11-07 18:00:09
Waiting for File Conversion             2005-11-07 17:56:54
Manuscript Submitted                    2005-11-07 17:56:54
Manuscript Files Submitted              2005-11-07 17:56:54
Preliminary Manuscript Data Submitted   2005-11-07 17:51:58

I just got a nice reply from Deleo about their GPS...

Open a new terminal window and type stty -f /dev/cu.usbserial 4800 or
-f /dev/cu.usbserial0 4800 depending on whether you used usbserial or
usbserial0 on the first window.  On the first window that opened you
will now start seeing incoming NMEA sentences which will look like
this: cat /dev/cu.usbserial


$GPGGA,212333.880,4738.3090,N,12219.2978,W,1,03,03.4,00080.6,M,-18.8,M,,*55
$GPGSA,A,2,,,,10,,06,,,,24,,,03.5,03.4,*1D
$GPGSV,2,1,07,17,30,238,,05,24,175,,25,18,286,,10,52,091,34*7A
$GPGSV,2,2,07,06,71,302,34,24,25,058,35,30,54,202,36*45
$GPRMC,212333.880,A,4738.3090,N,12219.2978,W,000.0,000.0,090303,018.6,E*71
$GPGGA,212334.880,4738.3088,N,12219.2978,W,1,03,03.4,00080.6,M,-18.8,M,,*5B

Which is a basic NMEA stream... why do all these things default to 4800 baud serial? So slow. At least it is better than the KLY-2 Kappabridge in the pmag lab which only does 300 baud.

Yeah! I don't have to break out GMT, Fledermaus, or Arc. Google Maps takes longitudes and latitudes. Now I can find that suspended sediment station quicker.

Update: Got a VERY quick response from John at the USGS. The Lon,Lats are indead ddmmss or dddmmss. The best relevant example of something like what I was thinking for a google mash-up is Wunderground stations. When their full database comes online, if the USGS does not do an interface, I would like to jump in and give it a go.



It is great that this data is online, but it is definitely confusing. Take a look at this sediment record, specifically at the lat,lon values. The lat,lon could use a little formatting help. It would be great to get this kind of info on a google mash-up web interface too! station id 11472150

Station Information:
Station Name:   EEL R NR DOS RIOS, CA
Station ID:     11472150
Basin Area (sq. mi.):   528
Latitude:       393730
Longitude:      1232025
         
Start Date:     01-Oct-1966
End Date:       30-Sep-1977
Days of Record:         4018
Days Missing:   0

And here is the table data:

Station Number  Date Water Year Discharge (cfs) Discharge (cms) Concentration(mg/L)     Load (ton/day)  Load (Mg/day)
11472150        01-Oct-1966     1967    4       0.113   1       0       0
11472150        02-Oct-1966     1967    3.8     0.108   1       0       0
...
11472150        20-Nov-1966     1967    4000    113     2630    28400   25800

Looks like there are some rounding errors.

Going by state, this is what I can find for the Eel. It is not obvious which station is the best.

11472150        EEL R NR DOS RIOS, CA
11472500        EEL R AB DOS RIOS, CA
11472800        MF EEL R AB BLACK BUTTE R NR COVELO, CA
11473000        MF EEL R BL BLACK BUTTE R NR COVELO, CA
11473900        MF EEL R NR DOS RIOS, CA
11474500        NF EEL R NR MINA, CA
11475000        EEL R A FORT SEWARD, CA
11475100        DOBBYN C NR FORT SEWARD, CA
11475500        SOUTH FORK EEL RIVER NR BRANSCOMB, CA
11475560        ELDER C NR BRANSCOMB, CA
11476500        SF EEL RIVER NR MIRANDA, CA
11477000        EEL R A SCOTIA, CA

I don't have the energy today to dig through this kind of database. Nor did the paper I read on this say which station ID they used.

This is a press release about the EPSL issue about the MER results. Now it is time for the research community outside of the MER team to step up with their take on the results. I waiting until after I graduate to get back into this topic.

Featured on Science Daily

RELEASE: 05-415


NASA Rover Helps Reveal Possible Secrets of Martian Life


Life may have had a tough time getting started in the ancient
environment that left its mark in the Martian rock layers examined by
NASA's Opportunity rover. The most thorough analysis yet of the
rover's discoveries reveals the challenges life may have faced in the
harsh Martian environment. 


"This is the most significant set of papers our team has published,"
said Dr. Steve Squyres of Cornell University, Ithaca, N.Y. He is
principal investigator for the science instruments on Opportunity and
its twin rover, Spirit. The lengthy reports reflect more thorough
analysis of Opportunity's findings than earlier papers. 


Scientists have been able to deduce conditions in the Meridiani Planum
region of Mars were sometimes wet, strongly acidic and
oxidizing. Those conditions probably posed stiff challenges to the
origin of Martian life. 


Based on Opportunity's data, nine papers by 60 researchers in volume
240, issue 1 of the journal Earth and Planetary Science Letters
discuss what this part of the Martian Meridiani Planum region was like
eons ago. The papers present comparisons to some harsh habitats on
Earth and examine the ramifications for possible life on Mars.


Dr. Andrew Knoll of Harvard University, Cambridge, Mass., a paper
co-author, said, "Life that had evolved in other places or earlier
times on Mars, if any did, might adapt to Meridiani conditions, but
the kind of chemical reactions we think were important to giving rise
to life on Earth simply could not have happened at Meridiani." 


Scientists analyzed data about stacked sedimentary rock layers 23 feet
thick, exposed inside "Endurance Crater." They identified three
divisions within the stack. The lowest, oldest portion had the
signature of dry sand dunes; the middle portion, windblown sheets of
sand with all the particles produced in part by previous evaporation
of liquid water. The upper portion corresponded to layers Opportunity
found earlier inside a smaller crater near its landing site. 


Materials in all three divisions were wet both before and after the
layers were deposited by either wind or water. Researchers described
chemical evidence that the sand grains deposited in the layers had
been altered by water before the layers formed. Scientists analyzed
how acidic water moving through the layers after they were in place
caused changes such as the formation of hematite-rich spherules within
the rocks. 


Experimental and theoretical testing reinforces the interpretation of
changes caused by acidic water interacting with the rock layers. "We
made simulated Mars rocks in our laboratory then infused acidic fluids
through them," said researcher Nicholas Tosca from the State
University of New York. "Our theoretical model shows the minerals
predicted to form when those fluids evaporate bear a remarkable
similarity to the minerals identified in the Meridiani outcrop." 


The stack of layers in Endurance Crater resulted from a changeable
environment perhaps 3.5 to 4 billion years ago. The area may have
looked like salt flats occasionally holding water, surrounded by
dunes. The White Sands region in New Mexico bears a similar physical
resemblance. "For the chemistry and mineralogy of the environment, an
acidic river basin named Rio Tinto, in Spain, provides useful
similarities," said Dr. David Fernandez-Remolar of Spain's Centro de
Astrobiologia. 


Many types of microbes live in the Rio Tinto environment, one of the
reasons for concluding that ancient Meridiani could have been
habitable. However, the organisms at Rio Tinto are descended from
populations that live in less acidic and stressful habitats. If
Meridiani had any life, it might have had to originate in a different
habitat. 


"You need to be very careful when you are talking about the prospect
for life on Mars," Knoll said. "We've looked at only a very small
parcel of Martian real estate. The geological record Opportunity has
examined comes from a relatively short period out of Mars' long
history." 

Digital Rights Management for the Geospatial Community - scary. Is this going to be like sony and their root kit?

What is the University Consortium for Geographic Information Science?

Q&A: Vint Cerf on Google's challenges, aspirations

  What do you make of the mashups phenomenon? I can't tell you how
  excited I am about it. We know we don't have a corner on
  creativity. There are creative people all around the world, hundreds
  of millions of them, and they are going to think of things to do
  with our basic platform that we didn't think of. So the mashup stuff
  is a wonderful way of allowing people to find new ways of applying
  the basic infrastructures we're propagating. This will turn out to
  be a major source of ideas for applying Google-based technology to a
  variety of applications. 

Wow. I typed twisted gps into the location field in firefox and got taken straight to this page!

twisted.protocols.gps.nmea.NMEAReceiver

This parses most common NMEA-0183 messages, presumably from a
serial GPS device at 4800 bps

No such luck with AIS. The protocols page does not show any protocols for AIS.

Auto-Carto 2006: Call for Papers - Heathman Lodge, Vancouver, Washington June 26-28, 2006.

Topics:

• yada yada web mapping yada yada
• Homeland security
• Geovisualization
• Geovirtual Environments
• Geospatial Data Models
• Geospatial Data Integration
• Spatial Data Mining
• Map Use and Design
• Spatial Cognition
• Projection and Transformation of Geographic Databases
• Spatialization
• Geographic and Cartographic Standards

What is spatialization?

I was talking to Alex about the idea of a community developed map based on input from people with GPS units. What do I run into just a week later, but exactly that. Open Street Map

  OpenStreetMap is a project aimed squarely at creating and providing
  free geographic data such as street maps to anyone who wants
  them. This is because most maps you might think of as free actually
  have legal or technical restrictions on their use, holding back
  people from all walks of life who would like to use a map for one
  reason or another.


  OpenStreetMap has created a cool online editing interface that
  allows anyone to upload their GPS tracks and contribute to the
  map. The data created can then be used in lots of different ways for
  free - something that isn't possible with commercial mapping. 

Question: GPS and the Mac

The Rayming TN200 USB GPS Receiver seems like a snazzy little device. I wonder how it really interfaces to the system. Would it be easy to integrate into any C or Python program on Mac or Linux? I really don't know what is in this driver: macdriverpl2300 1.0.8 zip. It can't be too bad, since gpsdrive says it works with the TripNav TN200.

Deluoe Electronics also has one that looks like it would work.

The GPS buyers guide did not seem to talk about what I am looking for. I want something pretty different for starters. I just want a USB device. No need for a screen to waste power. Then eventually, I want to get a hand help device that integrates with it.

One suggestion was the Macmap group on Yahoo.

The BU-303 USB GPS Receiver has WAAS, but is it usable?

One main point is that it looks like having WAAS support is a very good thing. The TN-200 says that it is WAAS-capable. What does that mean?

If a device is listed on the gpsd hardware page then it has got to be mot to bad to use, right?

On Nathan's suggestion I just gave ClamXav a try. Not bad. I still may end up going with an anacron job and/or checking out the daemon package in fink.

Also, I did a quick test with a demo virus file to make sure that the fink ClamAV is actually doing anything (I've never had it find anything). I used Nathan's suggestion for a test virus and got this:

open http://www.eicar.org/anti_virus_test_file.htm
# Down load the eicar.com test virus to my desktop
/sw/bin/clamscan --bell -r --log=/Users/schwehr/Desktop/virus-scan.txt \
     -i /Users/schwehr/Desktop/eicar.com
cat ~/Desktop/virus-scan.txt


========================================
Scan started: Mon Nov 28 13:42:05 2005


/Users/schwehr/Desktop/eicar.com: Eicar-Test-Signature FOUND


== summary ==
Known viruses: 41289
Engine version: 0.87.1
Scanned directories: 0
Scanned files: 1
Infected files: 1
Data scanned: 0.00 MB
Time: 3.130 sec (0 m 3 s)

Yeah! It actually did something. If I use anacron, I am thinking of something like this as the script.

/sw/bin/fink update clamav clamav-dev clamav-shlibs
/sw/bin/freshclam
/sw/bin/clamscan --bell -r --log=/Users/schwehr/Desktop/virus-scan.txt -i /
if [ $? != 0]; then
  somescript that notifies me obnoxiously if there is a virus found
  maybe use growl
  maybe touch "/Users/schwehr/Desktop/VIRUS_FOUND"
fi

MapServer Enterprise 0.9.1 download

  ... Feature Data Objects (FDO) technology and FDO providers. At this
  time providers for ESRI Shape and the Spatial Data File format are
  included. The FDO package must be downloaded and built prior to
  building MapServer Enterprise. 


  In a future site update, FDO will be a separate component. In
  addition, we will provide a more complete set of FDO providers
  including SDF, SHP, WMF, WSF, ODBC, ArcSDE, and more.
...

Online Sources of GPS Waypoints and Tracks

  MotionBased uses the open standard .gpx format, which is being
  embraced by more and more software manufacturers over time. An
  extensive list of online sources of .gpx data is available from
  TopoGrafix, the developer of EasyGPS and ExpertGPS. This list
  encompasses everything from dive spots to UK hiking trails. Travel
  by GPS, one of the sources on the list has a page posted on data I
  contributed--my Mt. Diablo circumnavigation. I haven't tried every
  source on the list, but GPXchange and TrailRegistry look
  particularly interesting.  

From Diez B. Roggisch: (and a similiar comment by Fredrik Lundh)

Use paramtetrized cursor.execute(..) That is instead of doing
  c.execute("insert into foo values ('%s')" % mytext)
do
  c.execute("insert into foo values (?)", mytext)
Attention, the actual style of a parameter is dependand on your
database, e.g. oracle uses a differnet one:
  c.execute("insert into foo values (:mytext)", dict(mytext=mytext))
The actual style to use is given in the docs, or can be queried with
connection.paramstyle.  I recommend reading the DB-API 2.0 specs.

PEP 0249 - Python Database API Specification v2.0 (07 Apr 1999)

SQLObject is a very exciting python package. It wraps Postgres, MySQL, and SQLite databases in a common API and provites a lot of help to make interacting with SQL databases more pythonic. From the documentation:

 db_filename = os.path.abspath('data.db')
 if os.path.exists(db_filename):
     os.unlink(db_filename)
 connection_string = 'sqlite:' + db_filename
 connection = connectionForURI(connection_string)
 sqlhub.processConnection = connection
...
 class Person(SQLObject):
     firstName = StringCol()
     middleInitial = StringCol(length=1, default=None)
     lastName = StringCol()

Causes a person table to be created:

CREATE TABLE person (
    id INT PRIMARY KEY AUTO_INCREMENT,
    first_name TEXT,
    middle_initial CHAR(1),
    last_name TEXT
);

There is a monster discussion on python eggs yesterday and today on python eggs and Debian. The only reason I am subscribed to that list is to try to figure out how to cope with eegs in fink. I still have no clue what to do about them. I need to see what is in dh_python. It would be nice to have a what to have a package say what dependencies it really has so that I can put them in the fink info file, but I do NOT want eggs to mess with other packages. That will break fink.

From: Josselin Mouette 
Subject: Re: [Distutils] formencode as .egg in Debian ??


Phillip J. Eby a ?crit :
> And over the last few months, I believe we've also succeeded in stomping
> most of the issues that people had with getting solid non-root
> installations on their Linux distributions.  So the reasons for developers
> to prefer their dependencies to be managed as eggs will only improve over
> time, as the egg system allows Python developers to control and introspect
> their dependencies, rather than keeping that information hidden behind
> diverse platform-specific packaging tools.


This is useful for non-root installations, but it is only a hassle when
you are making .deb's. Here, dependencies are already handled, and you
rely on the system to provide correct versions for packages you depend
upon.


> It's only "competitive" if you feel that there must be only one way to do
> it.  (And if you do feel that way, then it also should be obvious that eggs
> are the superior solution, since they don't take away any capabilities of
> the old, only provide new ones.)


They only introduce more complexity, instead of bringing real features.


> Obviously, every individual distribution would like to have Python packages
> conform to their individual system.  However, on the whole, it is clearly
> better for the Python developer to have practical dependency management
> that doesn't tie their efforts to a single platform, packaging system, or
> distribution.


And that's why there are things like dh_python to adapt python distutils
installations to be compliant to the Debian way of things. However, eggs
make it so complicated that the adaptation layer would be
unmaintainable. This is a good reason to think the whole egg system
should be avoided.

I also saw something about easy_deb in eggs.

Alias has been seriously sucking for a while and I stopped watching, so this does not surprise me.

...
  The spy drama starring Jennifer Garner will end its five-season run
  in May, ABC announced Wednesday, promising a big finish. 
...
  "Alias is not going to wind down as it comes to an end, it's going
  to rev up, and we're going to make it the event it deserves to be,"
  ABC Entertainment President Stephen McPherson said in a statement.
...

Alias

Here is a fun thing to try... I had a song that is on my iPod, but not on my laptop. I am far away from my music collection, su I would like to get it back on my laptop. Since iTunes will not let me drag from my iPod, I had to figure out where the actual file is stashed on the iPod. Apple has tried to make it hard to see where the files are by not doing an artist/album hierarchy. The trick is to play the song so that the mp3 file is open and use lsof (list of open files) to find out what file is open...

> lsof | egrep -i 'mp3|aac|m4a'
iTunes    15748 schwehr   21r  VREG       14,6   62808865     3996 /Volumes/myiPodRev2/iPod_Control/Music/F18/somesong.mp3

On Mac OSX in a xterm (you don't use terminal, right?!?), do this:

echo "hello" | pbcopy
touch foo.txt
open foo.txt
# TextEdit appears editing foo.txt
Apple-V
# hello is now pasted in you TextEdit window

Now do the reverse

touch foo.txt
open foo.txt
# Type this in TextEdit
This is textedit
# Highlight the above line and copy it
Apple-C
# Go back to your xterm...
pbpaste
# "This is textedit" appears in your terminal

So many times I find people don't realize that computers log much of what they do on the net. Other than luper worm entries, this one caught my eye today.

#   Hits        Files       KBytes      Visits    Hostname
18  471  0.54%  201  0.27%  335  0.00%  6  0.07%  netops-security-1.ucsd.edu

Chris, is that you?

I just restarted firefox doing an update a few days ago. The installation on restart of the browser was about 1/2 a minute total. The only thing that I don't like is that the about pane only says Firefox 1.5, when it is just 1.5 RC 2. How about an expert button? Even about Does not gove all that much info:

Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8) Gecko/20051111 Firefox/1.5

At least the release date is in there.

Thanks to Travis for a workaround with the f-distributions. There is a problem with array handling.

ipython
In [1]: import scipy.stats
In [2]: scipy.stats.distributions.f.ppf(0.95,2,9)
Out[2]: 0.0
In [3]: scipy.stats.distributions.f.ppf([0.95],2,9)
Out[3]: array([ 4.25649473])
In [4]: scipy.stats.distributions.f.ppf([0.95],2,9)[0]
Out[4]: 4.256494729093748

http://www.dotfunk.com/projects/weatherupdate/

  WeatherUpdate is intended to be used with Davis Monitor II, Oregon
  Scientific WM-918, and WMR-968. It manages all the communication to
  and from the weather station hardware and exports the current
  conditions to a varity of services and software.


  It can upload the current conditions to the Internet via via
  www.wunderground.com. Click here for more information about the
  Personal Weather Station Project. 


  It is written in Python.

For Perl, there is MisterHouse that also does home automation or wxnet.

Is there a home automation python library?

Now I just need some hardware and a some free time to give this a try!

Just released pmag-kds-py 0.11, which is mostly to compensate for scipy's f distribution code being broken (again).

• ams.py: Added getShapeName, getF12/23 now hardcoded(scipy bad)
• ams.py: Added getLparam, getFparam, getEta
• pmag_sqlhelp.py: Added sqlInsertStrFromList
• ttn136b.py: Added BadSamples class

This morning, I rebuilt my AMS graphs for my paper and got that all but a couple samples here trixial. What the? This is totally not what was going on last week. I logged into my server and did the sample thing there and got the old results with samples falling in every catagory and looking much more plausible. The F distribution code in scipy was broken for a very long time and recently started working. It looks like there may be a regression that happened at the same time I started getting those import printouts. Here is what I am seeing on my server that has not been updated. This is what I expect.

fink list scipy


 i   scipy-py24        1:0.3.2-2     Scientific tools for Python


import ams
>>> ams.getF()
3.4817
>>> ams.getF12()
4.25649472909
>>> import scipy.stats
>>> scipy.stats.distributions.f.ppf(0.95,2,9)
4.25649472909

Now for the broken results on my laptop:

fink list scipy
 i   scipy-core-py24       0.6.1-2          Replacement of Numeric python
 i   scipy-py24            1:0.4.3-1        Scientific tools for Python


In [1]: import ams
In [2]: ams.getF()
Out[2]: 3.4817
In [3]: ams.getF12()
Importing io to scipy
Importing fftpack to scipy
Importing special to scipy
Importing utils to scipy
Importing cluster to scipy
Importing sparse to scipy
Importing interpolate to scipy
Importing lib to scipy
Importing integrate to scipy
Importing signal to scipy
Importing optimize to scipy
Importing linalg to scipy
Importing stats to scipy
Out[3]: 0.0
In [4]: import scipy.stats
In [5]: scipy.stats.distributions.f.ppf(0.95,2,9)
Out[5]: 0.0

Time to hardcode the 0.95 values (again) and move on.

Scipy is acting cranky. I now get this when ever I import scipy. Very annoying.

Importing io to scipy
Importing fftpack to scipy
Importing special to scipy
Importing utils to scipy
Importing cluster to scipy
Importing sparse to scipy
Importing interpolate to scipy
Importing lib to scipy
Importing integrate to scipy
Importing signal to scipy
Importing optimize to scipy
Importing linalg to scipy
Importing stats to scipy

This is with fink scipy-core-py24 0.6.1-2 and scipy-py24 1:0.4.3-1.

I want to make a quick list of bad samples and make it easy to quickly test against this. Here is a quick walk through of sets, which are basically what I want.

  s1 = set()
  s1.add('foo')
  s1.add('bar')
  s2 = set()
  s2.add('blah')
  s2.issubset(s1)
False
  s2 = set()
  s2.add('foo')
  s2.issubset(s1)
True

Congress approves $8.1 billion for testing facility at PNS - The article says $8.1M which I believe more than billion.

...
  The funding will be used to construct a new acoustic test and
  calibration facility. 


  According to Rep. Tom Allen, D-Maine, on Friday the final report for
  the Fiscal Year 2006 Military Quality of Life and Veterans Affairs
  Appropriations bill was completed and it included funding for the
  shipyard. 
...
  A new acoustic testing and calibration facility will provide the
  Navy with a greater capacity to test and calibrate submarine and
  surface ship hydrophones and transducers, shipyard public affairs
  officer Danna Eddy said in a previous interview. 
...

3Dconnexion Boosts SpacePilot Performance for Autodesk Inventor

  LOS GATOS, Calif. - November 8, 2005 - 3Dconnexion, a Logitech
  (SWX:LOGN) (NASDAQ:LOGI) company, announced today a significant
  advancement to its innovative adaptive sensing technology, allowing
  the 3Dconnexion SpacePilot 3D controller to detect and rapidly
  adapt to more than twenty-one environments for Autodesk Inventor.
  As the user's workflow moves among environments and sub-environments
  within Autodesk Inventor, the SpacePilot automatically detects the
  workflow change, updates its function keys with new commands and
  displays them on its LCD screen. Default functions are provided for
  each environment, and the user can also customize the function keys
  by choosing from the entire set of Autodesk Inventor commands. 
...

Hey 3Dconnexion guys: How about announcing solid Space* support for SIMS Coin/SoXt/SoQt (SGI's OpenInventor done better than SGI did)?!?!?!

16 papers on real-time and embedded Linux [ linuxdevices.com ]

  LinuxDevices.com is pleased to publish the proceedings from the
  Seventh Real-Time Linux Workshop held in Lille, France, November
  3-4, 2005, at the University for Science and Technology of Lille
  (USTL). The papers span a broad range of topics, ranging from
  fundamental real-time technologies to applications, hardware, and
  tools. 

Too bad there are no audio versions that I could listen to on the shuttle. I was also bummed to see that most of the MIT open courseware has RealAudio streams but no mp3 or oggs. I would love to have the magnetics and linear algebra lectures to brush up with.

I am really not sure what I am looking at with this image... I think it needs a regional image and a second one with overlayed interpretation.

Landslide in Mutch [ MSSS ]

  This Mars Global Surveyor (MGS) Mars Orbiter Camera (MOC) image
  shows the east margin of a landslide off the southern rim of Mutch
  Crater in the Xanthe Terra region of Mars. This particular landslide
  was likely triggered by a meteor impact that occurred nearby. 

Myche just helped me work though building an SQL query to generate the data for a Banerjee plot. In plotting ARM100mT and IRM10kGauss, I have to pull the pairs out of different rows in the same table. Turns out this can be done with either temporary tables or subqueries. I started down the subquery path here, so that is what I will show.

First the schema for the table.

CREATE TABLE mag ( samplename VARCHAR(40), treatment REAL, csd REAL,
  intensity REAL, dec REAL, inc REAL, timeMeasured TIMESTAMP, type
  VARCHAR(10), operator VARCHAR(20), depth REAL, cruise VARCHAR(20),
  corenum INTEGER, coretype VARCHAR(1), corehalf VARCHAR(1) , section
  INTEGER, sectionoffset REAL);

The type field is either "arm" or "irm" for each measurement made on Flo the magnetometer. So I have a two rows, both with the same sample name that need to be joined together, but the intensity value for each must be renamed.

samplename         type  intensity
==========         ====  =========
ttn136b-2pw-s6-100 irm   0.0026260000
ttn136b-2pw-s6-100 arm   0.0000352900

To solve this I made a subquery for each for irm and arm samples,
renamed the fields and joined them by matching names.  This does leave
out the problem of separating cores from each other, but that is
pretty easy to add and will just make this example more confusing.

#!/bin/bash
#
q1="SELECT samplename AS armsample, intensity AS arm FROM mag WHERE type='arm'"
q2="SELECT samplename AS irmsample, intensity AS irm FROM mag WHERE type='irm'"
#
sqlite ttn136b.db "SELECT irm,arm FROM ($q1),($q2) WHERE armsample=irmsample;"

This results in this query getting sent to sqlite:

  SELECT arm,irm FROM \
    (SELECT samplename AS armsample, intensity AS arm FROM mag WHERE type='arm') \
   ,(SELECT samplename AS irmsample, intensity AS irm FROM mag WHERE type='irm') \
    WHERE armsample=irmsample;

If I were a big Java programmer, I would definitely check out GeoTools

Publications related to Google technology

Sawzall... location/time of google searches. First saw something like this at ATT in early 1999 on their big SGI, excpet the viz was in 3D and was telephone calls, not queries.

Saving the Net: How to Keep the Carriers from Flushing the Net Down the Tubes By Doc Searls

I have been wanting to write something along these lines.

  They see a problem with freeloaders. On the tall end of the power
  curve, those 'loaders are AOL, Google, Microsoft, Yahoo and other
  large sources of the container cargo we call "content". Out on the
  long tail, the freeloaders are you and me. The big 'loaders have
  been getting a free ride for too long and are going to need to
  pay. The Information Highway isn't the freaking interstate. It's a
  system of private roads that needs to start charging tolls. As for
  the small 'loaders, it hardly matters that they're a boundless
  source of invention, innovation, vitality and new business. To the
  carriers, we're all still just "consumers". And we always will be. 

Doc only scratches the surface. I have a lot more building up in my head. The issue is not just the carriers, but Google, Yahoo, and Microsoft are in the process of destabilizing (aka centralizing) the internet. Why I think that the map-mashups show the heart of the problem... I fear the essay I am planning is maybe 4 months away.

More and more things everyday point to the fact that I need to upgrade to Mac OSX 10.4. I am starting to get left behind with 10.3.9.

J2SE 5.0r3 for 10.4.2 or later.

In the strange ideas catagory, it would be interesting to see what is possible by using AIS signals for positioning of a receiver. Could use a combination of signal strength, direction, and timing info. This would be like the new receivers that do positioning from TV towers, but here the base stations are not FIXED TV broadcast towers, but now (mostly) mobile systems that are sending out their GPS location and velocity vector.

Volume 240, Issue 1, Pages 1-190 (30 November 2005) Sedimentary Geology at Meridiani Planum, Mars Edited by Steven W. Squyres and Andrew H. Knoll

• Sedimentary rocks at Meridiani Planum: Origin, diagenesis, and implications for life on Mars
• Stratigraphy and sedimentology of a dry to wet eolian depositional system, Burns formation, Meridiani Planum, Mars
• Chemistry and mineralogy of outcrops at Meridiani Planum
• Provenance and diagenesis of the evaporite-bearing Burns formation, Meridiani Planum, Mars
• Geochemical modeling of evaporation processes on Mars: Insight from the sedimentary record at Meridiani Planum
• The Río Tinto Basin, Spain: Mineralogy, sedimentary geobiology, and implications for interpretation of outcrop rocks at Meridiani Planum, Mars
• Hematite spherules in basaltic tephra altered under aqueous, acid-sulfate conditions on Mauna Kea volcano, Hawaii: Possible clues for the occurrence of hematite-rich spherules in the Burns formation at Meridiani Planum, Mars
• An astrobiological perspective on Meridiani Planum

How is a 33 page paper a "short communication"?

Just discoved another interesting BASH trick. If you are running bash as your shell, open up an xterm (or I suppose a Terminal.app window if you must) and try this:

    # echo $RANDOM
    31162

Returns a 16 bit unsigned integer (ushort). Never knew this before!

I know that lines of code is a horrible metric, but I felt like taking a 15 minute break to see how many lines of code are in my thesis that are written by me. There are a couple of big reasons why lines of codes sucks:

• I can do more in python in one line than with C (excepting macro craziness)
• There are a lot of embedded documentation strings
• What if I like or dislike whitespace lines?
• What if a project autogenerates files in certain languages? I do this for gnuplot in my thesis.

I did an egrep -v on these values:

SKIP:='sioseis|dac-cmds|pltsegy|xsonar|pmag-py|giv|\~|bak|xvpics|\,v$$|\.in|foo|Away'

I then ended up with this:

Total lines written:
       C++: 37385
    python: 22457
     Makes: 6343
      bash: 5594
    C code: 4768

The big surprise for me is how much straight c code I still have floating around. 75k lines of code is respectable for someone who is officially a geologist

I needed to get a summary of what data is available for my Eel River cores. Rather than go through the files by hand or with awk and grep, I thought I'd create an SQL program to crunch through all the possibilities. It turned out pretty well. I could do more by introspecting the database, but this is all that I need.

./summary.py 
1 p: {'hyst: 000, ams: 062, arm: 000, afdemag: 048, irm: 000'}
1 t: {'hyst: 000, ams: 000, arm: 000, afdemag: 000, irm: 000'}
2 p: {'hyst: 013, ams: 064, arm: 000, afdemag: 064, irm: 000'}
2 t: {'hyst: 000, ams: 055, arm: 000, afdemag: 002, irm: 000'}
3 p: {'hyst: 000, ams: 051, arm: 000, afdemag: 051, irm: 000'}
3 t: {'hyst: 000, ams: 000, arm: 000, afdemag: 000, irm: 000'}
4 p: {'hyst: 000, ams: 064, arm: 000, afdemag: 063, irm: 000'}
4 t: {'hyst: 000, ams: 000, arm: 000, afdemag: 000, irm: 000'}
5 p: {'hyst: 000, ams: 074, arm: 000, afdemag: 000, irm: 000'}
5 t: {'hyst: 000, ams: 000, arm: 000, afdemag: 000, irm: 000'}
6 p: {'hyst: 000, ams: 000, arm: 000, afdemag: 000, irm: 000'}
6 t: {'hyst: 000, ams: 000, arm: 000, afdemag: 000, irm: 000'}
7 p: {'hyst: 000, ams: 065, arm: 000, afdemag: 065, irm: 000'}
7 t: {'hyst: 000, ams: 008, arm: 000, afdemag: 000, irm: 000'}

Yes, I did a little bit of search and replace to make the table cleaner. Maybe I should add that to the code, but not today. Here is the python for the above summary.py.

#!/usr/bin/env python
"""
Generate a summary of the data collected to date
"""
import sqlite,sys
def counts(db_cx,corenum,coretype):
    results={}
    corenum = str(corenum)
    s = 'SELECT COUNT(samplename) AS count FROM '
    s+= '(SELECT DISTINCT(samplename) AS samplename FROM '
    s+= 'ams WHERE corenum='+corenum+' AND coretype="'+coretype+'"); '
    cu.execute(s)
    count = cu.fetchone()['count']
    results['ams']=("%03d"%int(count))
#
    s = 'SELECT COUNT(samplename) AS count FROM '
    s+= '(SELECT DISTINCT(samplename) AS samplename FROM '
    s+= 'mag WHERE corenum='+corenum+' AND coretype="'+coretype+'"'
    s+= ' AND type="afdemag"'
    s+='); '
    cu.execute(s)
    count = cu.fetchone()['count']
    results['afdemag']=("%03d"%int(count))
#
    s = 'SELECT COUNT(samplename) AS count FROM '
    s+= '(SELECT DISTINCT(samplename) AS samplename FROM '
    s+= 'mag WHERE corenum='+corenum+' AND coretype="'+coretype+'"'
    s+= ' AND type="arm"'
    s+='); '
    cu.execute(s)
    count = cu.fetchone()['count']
    results['arm']=("%03d"%int(count))
#
    s = 'SELECT COUNT(samplename) AS count FROM '
    s+= '(SELECT DISTINCT(samplename) AS samplename FROM '
    s+= 'mag WHERE corenum='+corenum+' AND coretype="'+coretype+'"'
    s+= ' AND type="irm"'
    s+='); '
    cu.execute(s)
    count = cu.fetchone()['count']
    results['irm']=("%03d"%int(count))
#
    corenum = str(corenum)
    s = 'SELECT samplename FROM hyst;'
    cu.execute(s)
    count = 0
    findStr = '-0'+corenum+coretype
    for row in cu.fetchall():
        if -1 != row['samplename'].find(findStr): count += 1
    results['hyst']= ("%03d"%int(count))
#
    return  results
#
if __name__=="__main__":
    cx = sqlite.connect("ttn136b.db")
    cu = cx.cursor()
    if True:
        for corenum in range(1,8):
            afor coretype in ('p','t'):
                print corenum, coretype,':',
                print '  ',counts(cx,corenum,coretype)

I have lots of measurements (we call them treatments) of samples in a database table. I would like to have a count of how many samples have gone through the process. A subquery using distinct passing to count will do the trick. Which one of these is more correct? They both give the same answer.

sqlite ttn136b.db
sqlite> SELECT COUNT(samplename) FROM (SELECT DISTINCT(samplename) AS SAMPLENAME FROM mag);
310
sqlite> SELECT COUNT(samplename) FROM (SELECT DISTINCT samplename FROM mag);
310

The top one looks more correct to me, but the bottom one is shorter. It would be great if sqlite would tell me if what I was doing was iffy SQL. Just because some code works doesn't mean that it is a good idea.

I posted on usenet (does anyone have a real nntp feed anymore or is everyone using google groups???) about gnuplot. I would like to have some variables that I can control from the command line or environment variables, but it seems like there is no joy. I did find out that 4.1 has looks of new features. One I am really hoping for is being able to place images into graphs. I have core photos and it seems silly to paste them in with Illustrator. "Automation good."

   I am not a number.
   I am not a number.
   I am not a number.
   I am a free man.

The computer, however, is not free from my wrath.

I have to admit that I really like the show 'Arrested Development'. I haven't really watched it on TV, but earlier this year, we had quite the marathon. "There is money in the banana stand!"

Fox puts abrupt stop to 'Arrested Development'

  There was no official announcement -- there rarely is when the
  networks wield the ax -- but after two and half wonderfully funny
  seasons, Fox's Emmy-winning ``Arrested Development'' is dead. 


  Network TV's best comedy has been abruptly pulled from the Fox
  schedule for the rest of November sweeps and has had its `order''
  for this season cut from 22 episodes to 13.

Jar Head... this movie is awesome! Just be warned that it really is not an action film.

I was wrong. Gareth just told me that the very cool program that he showed me a couple months ago is actually Regex Coach

My ucsd email seems to be unhappy, which makes me unhappy.

I am working on straightening out the data for the Eel River Basin (TTN136B) cruise data. I am working on adding the AGFM data to the database. Turns out I took some bad data yesterday. I need to take data over a larger range. The tails are really ugly. In the process, I have a new release of pmag-kds-py (0.10).

• agfmsql.py - Deleted this file
• agfm.py - Now has sql database support
• agfm.py - Much improved
• ttn136b.py - Able to revert samplenames to original
• units.py - Added incFromLat using GAD model

An amusing aside... here are the line counts for the project:

    910 agfm.py
    193 ams.py
    325 bpsio.py
    445 fabricsim.py
    128 pdfposter.py
     33 pmag_kds_version.py
    211 pmag_sqlhelp.py
     52 pmag_verbosity.py
     41 setup.py
    256 ttn136b.py
    228 units.py
    885 xcore.py
    379 xrd.py
   4086 total

Excuse me, but what exactly is it that these folks are selling? Is it bathymetry, or are they trying to imply more?

  SeaZone Hydrospatial represents an important sea-change in marine
  data by comprising a comprehensive, accurate and current map of the
  entire marine environment and coastal zone. Being interoperable with
  Ordnance Survey mapping, it effectively continues the digital
  national framework offshore. The launch of SeaZone Hydrospatial
  reinforces the UK's reputation as a world leader in geographic
  information collection and provision and will consign the use of
  navigational charts back to the ship's bridge.

Introduction to OOP with Python Object Oriented Programming [voidspace]

scipy05 conference presentations. Presentations on matplotlib, scipy, arrays, MPI, and more

I think I will stick to google maps and yahoo beta maps, but it is good to know what else is out there for that rainy day.

• http://www.map24.com/
• http://www.maporama.com/share/
• http://mappoint.msn.com/
• http://www.randmcnally.com/rmc/directions/dirMapsDirections.jsp

Web mapping is actually one item for a paper that I am thinking of writing: Why the internet is failing to live up to its design.

I just ran into a simple case of something that has worried me a bit about python. I have a function foo that use to be like this:

  def foo(arg1, argBoolFlag=False, argInt=0)

Now I change foo to have a 2nd flag and I put the new flag after the 1st.

  def foo(arg1, argBoolFlag=False, argBoolFlag2=False, argInt=0)

Now my older code does bad things. Here is the old call:

  badresults = foo(myarg1,True,1)

Not checking that the flag args were actually boolean (True/False) values bit me. This is fine now since I'm working on a program with about 1000 lines. Not to bad to track down in code all written by me. Now what happens in a couple hundred thousand line system written by a team of people?

On second thought, C++ would let me shoot myself the same way by casting an int to a bool without fuss.

http://planet.spatiallyadjusted.com/ - planet geospatial is an aggregation of weblogs and web papages about GIS and other spatial stuff.

I think I've just seen the spinning rainbow/pinwheel/pizza of doom on my laptop one too many times on my mac while using Firefox. I think I might just have to go back to Safari as my main browser.

I'm using 1.5 rc 1. Have to grab rc 2 which I hear is floating around.

top -o cpu -s 5
  PID COMMAND      %CPU   TIME   #TH #PRTS #MREGS RPRVT  RSHRD  RSIZE  VSIZE
  803 firefox-bi  15.0% 18:10.40   3   101   697  83.0M  48.9M- 99.9M+  332M

Okay, so I have a gig of ram, but there is no need to chew up so much.

I am feeling a little frustrated with plotting from within python today, so I went back to straight gnuplot. Somehow, it is just simpler. I found a web page that I really like so far for help with gnuplot. GNUPLOT - not so frequently asked questions.

#!/bin/bash
#
!echo "#depth bulbsusc tau1 tau2 tau3 dec1 inc3" > 2p.dat
!sqlite ttn136b.db 'SELECT depth, bulksusc, tau1, tau2, tau3, \
  dec1, inc3 FROM ams WHERE corenum=2 AND coretype="p" \
  ORDER BY depth' | tr '|' '\t' >> 2p.dat
#
unset key
set yrange [] reverse
set multiplot
########################################
# Bulk suscecptabilty SI
set title 'Bulk Chi (SI)'
set xtics .0001,.0002
set origin 0,0
set size .3,1
plot '2p.dat' using 2:1 with lp
########################################
# tau 1, 2, 3 - eigen values
set title 'tau 1,2,3'
set noytic
set xrange [.32:.35]
set xtic .33, .01, .34
set origin .25,0
plot '2p.dat' using 3:1 with lp \
     ,'2p.dat' using 4:1 with lp \
     ,'2p.dat' using 5:1 with lp
########################################
# dec1 eigenvector (with minimum eigenvalue)
set title 'dec 1'
set xrange [0:360]
set xtic 90,90,270
set origin .5,0
plot '2p.dat' using 6:1 with lp 
########################################
# inc3 eigenvector component (with minimum eigenvalue)
set title 'v 3'
set xrange [0:90]
set xtic 30,30,60
set origin .7,0
plot '2p.dat' using 7:1 with lp 
#
unset multiplot
pause -1 'hit return dummy'

This yields a very imperfect plot, but it is heading in the right direction. This is my first time really tweaking xtic to not clobber neighboring plot labels across the bottom.

Same guy as the pirate map... Justin's Yahoo Radar Map

Just released pmag-kds-py 0.9...

• fabricsim.py: Version used in Santa Barbara paper (angles backwards)
• pmag_sqlhelp.py: Added sqlcreate class
• ttn136b.py: Added depthlookup.lookup2()
• xrd.py: More documentation and a sample card. Need more info

For those in the know, I am sure that this is old news, but I just discovered that ERSI's ArcGIS supports the python language. However, it still is not available on Mac OSX and I can't find (in 20 seconds) a good summary page on Arc/python that is not a pdf. Someone know of the canonical non-pdf Arc/python web page?

Introduction to Geoprocessing Scripts Using Python - ESRI class on python

slashgisrs

Worldwind has been forked as punt

Don't blame the online mappers

  reporter's notebook For the record, the guys who create those
  often-vexing online maps would like you to know it's not their
  fault. 


  Or so they say. On a recent sunny Thursday morning, I went directly
  to the source to find out why online maps more than occasionally
  have bizarro directions that would make any good cabbie scoff. 


  Taking a leisurely drive around San Francisco's upscale Nob Hill
  neighborhood in a video camera-equipped van owned by Tele Atlas (one
  of the two main mapping data collectors), I realized that it's
  awfully hard to argue with video evidence. Tele Atlas has a van
  whirring with videos, computers and navigation equipment, capturing
  up-to-date navigation information about my city.  
...

What we need is an open source mapping system for PDA's with embedded GPS units + a system for anonymizing who you are so the feds don't get to know your every move for free. If everyone had a "bitch" option to complain when directions were wrong or missing, would we have a better system? What if everyone had free access to the whole database? What could be created?

Rick Rashid - Microsoft Research - ETech 2005

  Among the topics of discussion are technologies that would allow
  users to capture literally everything that happens to them, not just
  visually, but with other associated meta-data. Another interesting
  area of research is technology that can make almost any physical
  object into a computer interface and can create digital
  representations of objects that users can manipulate in the digital
  environment. mp3

Think about this kind of device on a field geologist or someone working on a ship processing through a very large volume of material. This stuff is pretty scary from a privacy point of view, but the applications where that is not a problem (especially when there is a don't record button and an obvious "RECORDING" indicator) are enormous.

Whoops. I think the latest script for configuring fink mirrors has a slight bug. I don't think any of these countries are in North America.

Choose a continent:
(1)     Africa
(2)     Asia
(3)     Australia
(4)     Europe
(5)     North America
(6)     South America, Middle America and Caribbean
Your continent? [5] 
Choose a country:
(1)     No selection - display all mirrors on the continent
(2)     China
(3)     Georgia
(4)     Hong Kong
(5)     Indonesia
(6)     India
(7)     Japan
(8)     South Korea
(9)     Malaysia
(10)    Philippines
(11)    Saudi Arabia
(12)    Singapore
(13)    Thailand
(14)    Taiwan
Your country? [1] 7

I figure that Japan is the closest/fastest network of the bunch to San Diego

It would be really nice to have some sort of tool that helped people pick mirrors. Maybe something that figured out who is geographically closest and has low ping times? I have no idea where many of these mirrors are located and how good their network drops are.

Because of my last post about subversion (svn) I remembered that a while back I was unable to get php5 working on my laptop to try out some simple php again. I got totally frustrated back then. Configuring apache is worse than using the command line configure for a linux kernel. But sometimes you just stumble into the right thing at a later date. Here is what I had before that DID NOT work.

AddHandler php-script php
AddType text/html       php
AddType application/x-httpd-php-source phps

Then I ran into this very verbose (which is a good thing) description of installing apache2 on fink: Jay's how to install apache2 on fink. Based on his web page, I changed my /sw/etc/apache2/httpd.conf to have these lines.

DirectoryIndex index.html index.php


AddType application/x-httpd-php .php .php3 .php4 .php5 .phtml
AddType application/x-httpd-php-source .phps

Now php5 is working. Woo hoo! Now I just need time to play with php some. I also probably need to read up on php security, but my laptop's webserver never sees the internet, so I can put that off.

Getting Control with Subversion and Xcode [apple.com]

This article talks about setting up svn and apache 2.0 on Mac OSX 10.4. I would like to remind people that if they use fink (or darwinports) to install these two systems, they can stay up to date much easier and should never have to patch anything. I can't wait to switch from CVS to SVN once I finish my doctoral dissertation!

yahoo pirate maps - I can't seem to find the burried treasure's location on the map.

http://glintercept.nutty.org/

  Save all OpenGL function calls to text or XML format with the option
  to log individual frames. See here for an example of a single
  frames' XML logging.

Talking to someone looking to dump some geometry. The alias wavefront obj format is pretty dirt simple to write, but terrain-0.1 did not have the xc-obj.[CH] files, so now terrain-0.2 is out with that. You are welcome to use this code under the GPL. The code is pretty low on documentation.

http://schwehr.org/software/terrain-0.2.tar.bz2

blender3d will read OBJ files.

This script compares the references of figures to the label definitions of figures. These need to match up in a paper! The key is to use findall().

#!/usr/bin/env python
import re
#
figRE = re.compile(r"\\ref{fig:([a-z]+)}")
figDefRE = re.compile(r"\\label{fig:([a-z]+)}")
#
cites = {}
labels = {}
#
for line in open('short.tex').readlines():
    matches = figRE.findall(line)
    if None!=matches:
        for key in matches:
            if cites.has_key(key):
                cites[key] = cites[key]+1
            else:
                cites[key]=1
    matches = figDefRE.findall(line)
    if None!=matches:
        for key in matches:
            if labels.has_key(key):
                labels[key] = labels[key]+1
            else:
                labels[key]=1
#
citesSet=set(cites)
labelsSet=set(labels)
diffSet = citesSet.difference(labelsSet)
if len(diffSet)>0:
    print 'ERROR: missmatch between cites and labels...'
    for key in diffSet:
        print '  ',key

3Dconnexion SpacePilot review

...
  Primarily, the SpacePilot was designed with the intention of being
  used with 3D design software to manipulate perspectives and objects
  in three-dimensional space. In this type of application, the
  controller cap acts as an analogue to the axes of the virtual 3D
  world. Pulling, prodding, and manipulating the cap directly
  correlates to the movement of the camera or a 3D object. 


  I spend part of my time doing 3D design, and I am already quite used
  to the limitations of manipulating 3D space on a 2D plane in
  software such as Maya or 3ds max. These software packages have,
  since their inception, relied on the user's savvy with the
  simultaneous use of the keyboard and the mouse to provide the
  maximum interface flexibility possible. Even then, many users
  traditionally rely on the use of the three-button mouse in order to
  help translate the movement of the mouse on a flat surface into
  movements in space. But the act of "toggling" such movements is
  never a smooth one.  
...
  Frankly, I love this thing. It's well-designed, well-built,
  well-implemented, and, well, works well!  

I need to do a little python regular expression stuff to repeatedly crank through some documents to make sure that the figure reference and figures matchup. I was getting very frustrated that my matches were constantly empty. The trick is NOT to use the match(lineStr) method. That always to tries to find the match at the beginning of a line. Grrr. The correct method to use is search(lineStr). I am sure that match() is faster, but ouch. I would be so unlucky as to find a match() example first.

I am checking out some GUI regex tools since they make life easier when building re's. The first one I tried is kiki which comes as a part of SPE (Stanley's Python Editor). SPE is pretty rough around the edges, so I don't use that but kiki can be used by itself. Problem is that the fink package does not directly allow that to be done. i emailed the maintainer, but until then, here is how to use it directly.

sudo -s
cat <<EOF > /sw/bin/kiki.py2.4
#!/bin/bash


/sw/bin/python2.4 -OO /sw/lib/python2.4/site-packages/_spe/plugins/kiki/kiki.py
EOF
chmod +x /sw/bin/kiki.py2.4

That is ugly, but it does start kiki. wxGtk will complain about poll() being misused and kiki will eat way too much CPU/battery has it pummels the machine, but it does work. How do I make the fonts larger so I can read the text?

The next one is kodos which is based on PyQT. I think this is the one that Gareth showed me. It is much slicker than kiki, but it will need some attention before it can go in fink (it has some path problems).

  oh and what does segmentation fault mean? I got the program to
  compile and then when I tried to run it I got that error

Here is a quick sample of dealing with a seg fault. Start emacs and enter this BAD program as foo.C.

int main() {
  int v[3];
  v[100000]=-1;
  return(0);
}

Then build the program with debugging enabled (-g):

M-x compile
g++ -g foo.C -o foo

Run the program

M-x compile
./foo
#  ./foo
#  Compilation segmentation fault at Tue Nov  8 09:57:51

Whoops... something bad happened. Start the debugger.

M-x gdb
gdb --annotate=3 foo

Now use the debugger to see what is wrong. "(gdb)" is the debugger's prompt.

(gdb) break main
(gdb) r   # a.k.a "run"
Starting program: /Users/schwehr/Desktop/foo 
Reading symbols for shared libraries . done
#
Breakpoint 1, main () at foo.C:3
(gdb)

In the foo.C buffer, you should see a red circle on the line that the program is at.

(gdb) print v[0]
$1 = 0  # Only in -g mode with variables be initialized to zero (I think)
(gdb) n   # a.k.a. "next"
#
Program received signal EXC_BAD_ACCESS, Could not access memory.
0x00001dec in main () at foo.C:3
(gdb) quit

That is it for the quick tutorial.

Computer Hardware Understanding Development Tools (CHUD Tools)

  The CHUD Tools are applications and tools for measuring and
  optimizing software performance on Mac OS X as well as for hardware
  bringup and system benchmarking. For the very latest version, and
  for Project Builder/Jaguar developers, the CHUD Tools can be
  downloaded from this ftp site.

History's Worst Software Bugs

This is not quite the list that I would give, but there are some good ones. Would have been nice for them to give more links to actual details.

Trincardi and Syvitski have a new volume in Marine Geology that starts off with:

Advances on our understanding of delta/prodelta environments: A
focus on southern European margins

Today, I am working on the final touches before resubmitting my Santa Barbara Basin paper. This is the standard header that I use for LaTeX typesetting of my paper.

\documentclass[agupp]{aguplus}

For actually submitting the paper, it has to be in a very funky special style that does double spacing and shoves my tables to the end of the paper. The submission format requires this first line instead of the one above.

\documentclass[agums]{aguplus} % For review - double spaced

It does not seem right to keep editing the paper for each format. I decided to create another mode much like I have for creating a copy without the figures (so it is much smaller). Here is the submit rule from my Makefile:

submit:
        echo "\documentclass[agums]{aguplus}" > ${NAME}-submit.tex
        egrep -v 'documentclass|\$Id:' ${NAME}.tex |perl -p -e "s/bibliography{santabarbara}/bibliography{santabarbara-submit}/g"   >> ${NAME}-submit.tex
        cp ${NAME}.bib ${NAME}-submit.bib
        pdflatex ${NAME}-submit.tex
        bibtex ${NAME}-submit.aux
        pdflatex ${NAME}-submit.tex
        pdflatex ${NAME}-submit.tex
        rm -f *-submit.{aux,bbl,blg,log,tex}

The first each creates the new header in a file. Then I append the original tex file but strip the old documentclass and the RCS/CVS Id, and change the bibtex file name (why do I have to do this?)

So who in San Diego has the Honda with the license plate "Mars Ops?" Is it someone at MSSS?

This is my first time using the '!' shell escape command in a gnuplot file to generate the datafiles for a plot from within the gnuplot command file (flinn-jelinek.bash). I have not yet done the Jelinek plot, so just the Flinn plot here. I feel pretty clever right now

#!/usr/bin/env gnuplot
# Build Flinn and Jelinek plots
#
#####################
# FLINN
#####################
#
# Build the data files for the plot
! sqlite ~/x/data/BPSIO-04/bpsio04.db 'select tau2/tau3,tau1/tau2 from ams where corenum=1' | tr '|' ' ' > 1.fl
! sqlite ~/x/data/BPSIO-04/bpsio04.db 'select tau2/tau3,tau1/tau2 from ams where corenum=2' | tr '|' ' ' > 2.fl
! sqlite ~/x/data/BPSIO-04/bpsio04.db 'select tau2/tau3,tau1/tau2 from ams where corenum=4' | tr '|' ' ' > 4.fl
# Flinn diagram is F vr L
set ylabel 'T1/T2 (L)'
set xlabel 'T2/T3 (F)'
#
set label 1 'Oblate' at 1.06, 1.04
set label 2 'Prolate' at 1.04, 1.06
#
set xrange [1:1.1]
#
set title 'Flinn Diagram'
f(x) = x
plot  '1.fl'  title 'Core 1'\
     ,'2.fl'  title 'Core 2'\
     ,'4.fl'  title 'Core 4'\
     ,f(x)
pause -1 "yo, press enter to move on with life"

I believe AIS has info in the streams to self identify when a ship is in distress, but what about designing a network to watch ships along the coast? It would be possible to flag ships that were acting outside of their normal paramters or just flat dropped off AIS for too long. For example if the crew was incapacitated, it would start to be clear that the ship was adrift or not being steered. Matt mentioned that people are looking into tracking AIS for monitoring pollution and whale issues.

This idea follows after the NASA Ames Code-IC work to make a system for monitoring the health of spacecraft (specifically Hubble) and notify someone if unexpected events occured.

Coast Guard rescues 4 at sea

...
  Petty Officer Dean LaMontagne said the Seawitch V, based out of
  Portsmouth, N.H., was located 95 miles off the Maine coast when it
  was hit by a large wave, which broke windows and flooded out the
  inside of the vessel. 


  "They lost all electronics on the boat except for the radio,"
  LaMontagne said. "The vessel couldn't go anywhere."
...

It never ends, especially when I have a whole bunch of feeds in netnewswire. And the reduced traffic side, I think that digg is starting to loose it's edge. Too many repeats.

dshield.org - See if your machine has been owned and is being bad.

Port Authority - DarwinPorts GUI (ala fink commander). Then there is A bash script to automate DarwinPorts upgrades

Create customized weather satellite movies [macosx hints]

Yahoo Research and Yahoo Next

http://charlespetzold.com/etc/DoesVisualStudioRotTheMind.html

How Stuff Works - google-earth

Tasting Menu - Online cookbook - Japanesse.

EchoFX - USB video digitizers for the Mac.

SVK is decentralized version control system. It uses the Subversion.

Recent Landslides In La Conchita, California Belong To Much Larger Prehistoric Slide

Simpleview flash gallery generator.

Minix3 is out. Dave gave me a copy of the official minix distribution back in 91/92'ish. Minix3 sounds more interesting than previous versions... still not all that exciting.

It seems silly to use spotlight for some of the search ideas that I have been pondering. I like the idea of being able to target swish-e right at just the data that I want searched which is especially important if I intend to expose the search to the web. Also, swish-e is cross platform and open source. I never got the perl API working with fink, but I am excited to give it a try with the python API.

SwishE python API.

View the Moon in 3D on Your Desktop [slashdot]- NASA's World Wind does clementine data. I remember when Deanne Tucker did that for the Clementine data back in 1998 or so.

Yahoo!'s new twist on mapping APIs [Simon Willison's Weblog] - Never heard of Flex. Lots of different Yahoo API's.

SGI Powers San Diego State University's 3D Geospatial Mapping to Fuel Hurricane Relief Efforts... SGI's last gasps. Geomatrix, Eric Frost, CalIT2, and many of the usual cast of characters. But now NASA Ames. I don't really see the need for 10GigE... Err Ethernet is for LANs not WANs and we still have not figured out how to effectively use 10 or 100Mbit lines. P2P would be a big help here. Lots of local fast connections (like near by cities) would get critical data to endpoints faster. Maybe some day I will get to check out SGI OpenGL Vizserver.

Yet another package that needs to be in fink: OSSIM... after gdal-py, openev, and helping to finish up PyGMT (and and and)

Can remember why I through this had relevance to my future research, but it certainly is a purdy picture... http://www.aaronkoblin.com/work/faa/. Oh, yes, we need to do this for shipping tracking on the oceans. It's going to be an inverse of most of the flight traffic.

Becca sent me a link to the new snazzy browser Flock

I highly recommend listening to this 20 minute podcast. A lot of the stuff I learned about design in Stanford's ME101 came from IDEO, Koosh, and Apple. I definitely want to get ahold of Tom's books

Tom Kelly, Managing Director, IDEO - Tech Nation

  Dr. Moira Gunn interviews Tom Kelley, author of "The Ten Faces of
  Innovation -- IDEO's Strategies for Beating the Devil's
  Advocate. Tom is the managing direcor of IDEO, that innovative
  design firm known worldwide for its originality.

Just saw the Rosetta Stone for Unix and saw an interesting Mac OSX command. Not a clean as hinv on IRIX (like that was clear), but it does say a lot about your mac... There is also sysctl, bless, pdisk, ifconfig, otool, ktrace, kdump, and the netinfo stuff. Never forget lsof (list of open files) for those bad days. I do miss the IRIX 'par -SS' command.

# nvram -p


yada yada


# ioreg -bls 


+-o Root  <class IORegistryEntry, retain count 11>
  | {
  |   "IOKitBuildVersion" = "IOKit Component Version 7.9: Wed Mar 30 20:07:52 P$
  |   "IONDRVFramebufferGeneration" = <0000000200000002>
  |   "IOConsoleUsers" = ({"kCGSSessionUserNameKey"="schwehr","kCGSSessionConso$
  |   "IOKitDiagnostics" = {"Container allocation"=2819069,"IOMalloc allocation$
  |   "IORegistryPlanes" = {"IODeviceTree"="IODeviceTree","IOUSB"="IOUSB","IOSe$
  |   "IOMaximumMappedIOByteCount" = 536870912
  |   "IOCatalogue" = ({"IOProviderClass"="IOPlatformExpertDevice","IOProbeScor$
  | }
  | 
  +-o PowerBook5,4  <class IOPlatformExpertDevice, registered, matched, active,$
    | {
    |   "IONWInterrupts" = "IONWInterrupts"
    |   "display-config-info" = <0000000000000000>
    |   "IOPlatformArgs" = <00cd600000ccf0000000000000000000>
    |   "system-id" = <"0000000000000">
    |   "AAPL,add-fcode-file" = <ff865c30>
    |   "name" = <"device-tree">
    |   "#size-cells" = <00000001>
    |   "IOPlatformSerialNumber" = "blah blah"
    |   "device_type" = <"bootrom">
    |   "scb#" = <00000001>
    |   "powertreedesc" = ({"service"="IOPMUSBMacRISC2 is not serializable","de$
    |   "graphics-setagressiveness" = <>
    |   "model" = <"PowerBook5,4">
    |   "customer-sw-config" = <"           ">
    |   "clock-frequency" = <09eb18ef>
    |   "serial-number" = <yeah, right...>
    |   "compatible" = <"PowerBook5,4","MacRISC3","Power Macintosh">
    |   "AAPL,phandle" = <ff889be8>
    |   "pid#" = <00003144>
    |   "#address-cells" = <00000001>
    |   "copyright" = <"Copyright 1983-2004 Apple Computer, Inc. All Rights Res$
    | }
    | 

MSSS just released this picture from Mars saying it is a land slide. This is my quick interpretation. Are there more lobes on either side? Or is this just generally a scree slope?



The original MSSS web page: Ophir Landslide - MGS MOC Release No. MOC2-1271, 4 November 2005

I was told yesterday that someone believed that a mac osx box had been infected with the bird flu virus. I can find nothing on the web to say that the bird flu virus can infect a Mac OSX box... nor any other virus. I have also concluded that the mcafee virus database is really lame. Not much in the way of useful information in there guys.

Please if anyone knows of a confirmed mac osx box infected with a virus, please let me know. I still believe in NOT putting things like Sofos and Norton AntiVirus on your mac. If you want, scan the machine with ClamAV once in a while... it may find viruses sitting in your email files, but those viruses won't do anything to your osx or linux machine (as far I know).

ClamAV - The free Anti Virus solution for Windows on Linux

  fink install clamav
  clamscan --bell -r --log=~/Desktop/virus-scan.txt -i /
  less ~/Desktop/virus-scan.txt

Bird flu virus infects PCs [zdnet]

Attackers are exploiting fears over bird flu by releasing a computer
virus attached to an e-mail passing itself off as containing avian flu
information, warned computer firm Panda Software.


According to Panda, the virus Naiva.A masquerades as a word document
with e-mail subject lines such as "Outbreak in North America" and
"What is avian influenza (bird flu)?"


When the file is opened, the virus modifies, creates and delete
files. A second part of the virus installs a program that allows
hackers to gain remote control of infected computers.

Check out http://maps.yahoo.com/beta/index.php ... way better. You can now drag around the map, there are scale bars, and there is a photoshop like navigation overlay window.

pyinstant reminds me of inlining assembly in C, accept now it is inlining C in python. Can I inline assembly in C and then in Python?

Just so you paleomagnetists in the crowd can readup about your magnetometers, here is the wikipedia entry on SQUIDs. I first tried typing squid into the search field and only got the animal.

I would appreciate it if someone would invent a high temperature SQUID that has the better noise characteristics of the low temperature SQUIDs! That would probably reduce the cost of these magnetometers.

pmag-kds-py 0.8 is out...

• fabricsim.py: Added program to draw clay particle cartoons
• ams.py: F-test improvements... needs testing/validation
• units.py: Added deg2rad() and rad2deg()

I though SGI's were the coolest computers everywhere (from 1992-1999) then I started wondering why I was spending so much money on the box and then on the support and the OS started not working that well. They sold OpenInventor, destroyed VRML, but my finally straw was the pathetic support of open source software and software installation in general. "inst" and the freeware CDs were horrible.

Still, the computers back when will live in my memory forever. How many hours did I spend in front of an IRIX box? My first Octane absolutely rocked!

So today, SGI got delisted from the stock exchange: SGI Delisted, Becomes Penny Stock

Too bad the whole IRIX deal from the kernel to inventor/explorer/performer wasn't opened up. Someday I am going to give one of the solaris bootable CD's a try... schillix or belenix. Just to see how they are doing. I have no intention of switching to Solaris!!!

I though I had an awesome package in matplotlib and have been hoping to start decreasing my useage of gnuplot. Gnuplot is pretty rugged, but is starting to feel its age. So why am I whining? I am currently working a little program to make cartoons of clay sediment fabric. I'd like to make a cute little figure to show what I see going on in my cores with overpressure. This is similar to what Lisa Tauxe got Rodrigo Coppelli do in his masters thesis. I really like what he did in Figure 29. The program is coming along well and the figure looks great on the screen. I decided it is time to put the figure in my paper. I dumped the matplotlib figure to an svg and brought it up in Illustrator like this:

fabricsim.py --noisy --plot-file=fabric.svg \
    --angular-dist=range_uniform \
     --max-angle=80 --min-angle=7 \
     --start=10 --end=30


open -a /Applications/Adobe\ Illustrator\ CS/Illustrator\ CS.app fabric.svg

It looks great. I am writing out a separate plot command for each line. This has a terrible side effect. For each plot command, it looks like there is a background box draw for the whole figure size. It is transparent, but there are something like a thousand of these things. OUCH! Guess I have to dump the lines to a gnuplot format file and make an SVG from there. How do I write a whole bunch of discreet lines in matplotlib with one plot command?

I think I am also having issues with the randomness of random.random. I am using /dev/random for the seed, but that is not going to improve the non-randomness of the normal random functions. Suppose I could use one from GSL or just keep reading from /dev/random and pay a speed hit.

I came up with my own little way to deal with optparse choice items. This lets me specify the choices once and then make a list of options for the user. The for loop builds a string that contains all the choices. I was surprised not to see anything that already handled this in optparse. If it is there, please someone tell me what it is!

    densityChoices = ('linear','exp','const')
    choicesStr = ''
    for i in range(len(densityChoices)): choicesStr+=densityChoices[i]+', '
    choicesStr=choicesStr[:-2]
    myparser.add_option('--density-type',dest='densityType',
                        type='choice',choices=densityChoices,
                        default='linear',
                        help='What method to use to generate density '
                        +'[default: %default] [choices: '+choicesStr+']')

There is a new paper out in Marine Geology about the landslides in the Santa Barbara Basin area.

Geology and tsunamigenic potential of submarine landslides in Santa Barbara Channel, Southern California by Fisher, Normark, Greene, HJ Lee, and Sliter.

  A large submarine landslide complex and four small landslides
  developed under the Santa Barbara Channel, suggesting a potential
  hazard from landslide-generated tsunamis. We integrate offshore
  stratigraphy and geologic structure, multibeam bathymetric
  information, and several kinds of seismic-reflection data to
  understand how and when the submarine landslides
  formed. Seismic-reflection data show that mass failure along the
  slope began at least 200 ka ago. Landslides appear as zones of poor
  reflectivity having an irregular upper surface, and these zones
  alternate vertically with strong parallel reflections. The
  emplacement ages of two of the three main landslide lobes are well
  established at 8 and 10 ka. The source material for the youngest
  part of the landslide complex was sediment of probable late
  Pleistocene and Holocene age that accumulated in a shelf-edge
  delta. Directly under this delta, growth of faults and anticlines
  was particularly intense and tended to oversteepen the deltaic
  deposits. These active structures also formed migration pathways and
  reservoirs for aqueous and hydrocarbon fluids from the deep
  basin. Tsunami deposits have been described from a low-lying area
  near Santa Barbara, and numerical modeling of tsunamis generated by
  hypothetical landslides in Santa Barbara Channel indicates a
  moderate to severe threat [Borrero, J.C., Dolan, J.F. and Synolakis,
  C.E., 2001. Tsunamis within the eastern Santa Barbara
  Channel. Geophys. Res. Lett., 28(4): 643-646.], involving wave
  runups of 2-20 m, for a range of assumed landslide
  volumes. Inundation from these waves, however, is expected to be
  highly focused so that only narrow (not, vert, similar10-km)
  sections of the shoreline would be affected. 

It seems that yesterday, I was unable to get nanoblogger 3.1 to get a posting to show up in the archives. Hopefully that was just a one day thing. So this is a test to make sure that it is back in working order.

Well, all is working now and pardon the double article. I copied my 31st article to the 30th and now both are in the archive. Strange.

If I were to write a firefox plugin, my first one would be to enable C-k, C-a, and C-e emacs style shortcuts! Not having them is driving me crazy.