05.31.2005 19:52
segy sql database working
Just packaged up segy-py-0.3.tar.bz2 for release... sql database generation.
Doh! I just spent 15 minutes writing a nice explanation of the database building of SEGY header information (both file and trace). Then I hit the wrong key in the nanoblogger add and nuked the whole thing. Basic gist... it works. It is heavy on disk usage and cpu cycles, but I wrote it in less than an hour and a half while on the airplane. It needs better typing as I assumed everything should be saved as a 4 byte integer, but it is a great first pass.
Doh! I just spent 15 minutes writing a nice explanation of the database building of SEGY header information (both file and trace). Then I hit the wrong key in the nanoblogger add and nuked the whole thing. Basic gist... it works. It is heavy on disk usage and cpu cycles, but I wrote it in less than an hour and a half while on the airplane. It needs better typing as I assumed everything should be saved as a 4 byte integer, but it is a great first pass.
ls -l *dbThe sample db creation code:
-rw-r--r-- 1 schwehr staff 18501632 May 31 19:40 bpsegy.db
sqlite bpsegy.db 'select x/3600.0,y/3600.0 from segyTrace' | head
-120.101111111111|34.4061111111111 -120.101111111111|34.4061111111111 -120.101111111111|34.4061111111111 -120.101111111111|34.4061111111111 -120.101111111111|34.4061111111111 -120.101111111111|34.4061111111111 -120.101111111111|34.4061111111111 -120.101111111111|34.4061111111111 -120.101111111111|34.4061111111111 -120.101111111111|34.4061111111111
#!/usr/bin/env python from segy import Segy import sqlite import osMy quick and dirty sql string generation code. Note that there are only INTEGER fields!
if __name__ == "__main__": dbName = 'bpsegy.db' if os.access(dbName,os.W_OK): os.remove(dbName) cx = sqlite.connect(dbName) cu = cx.cursor() xstar = Segy('bpsio01L00.xstar')
# # CREATE the database tables # cu.execute(xstar.header.sqlCreate()) if True: t = xstar.getTraceHdr(1) cu.execute(t.sqlCreate()) cx.commit()
# # POPULATE the tables # cu.execute(xstar.header.sqlInsert()) for i in range(1,xstar.getNumberOfTraces()): t = xstar.getTraceHdr(i)
if 0==i%500: x = t.getHeader('X')/3600. y = t.getHeader('Y')/3600. print i,': ',x,y cu.execute(t.sqlInsert(traceNumber=i)) cx.commit() # Flush this to the db!
... def sqlCreate(self,specificKeys=None, table='segyFile'): """ Build an SQL string to create a table for the keys/fields in a SEGY file headerFeels strange to have the sql string generators inside the header instances, but that way, I know the driver is loaded.
arguments: specificKeys - in set, these are the keys that will used in the create string table - SQL table name (may have different tables for different formats) """ if not specificKeys: specificKeys = self.drv.binaryHeaderEntries.keys() cStr = 'CREATE TABLE '+table+' (' for key in specificKeys: cStr += ' '+key+' INTEGER,' cStr = cStr[:-1] cStr += ');' return cStr
def sqlInsert(self,specificKeys=None, table='segyFile'): """ Build an SQL string to INSERT the values for the current file """ if not specificKeys: specificKeys = self.drv.binaryHeaderEntries.keys() iStrStart = 'INSERT INTO '+table+' (' iStrEnd = ' VALUES ('; for key in specificKeys: iStrStart += key+',' iStrEnd += str(self.getBinEntry(key))+',' iStrStart = iStrStart[:-1] + ')' iStrEnd = iStrEnd[:-1] + ');' return iStrStart + iStrEnd
05.31.2005 15:48
segy-py setup.py and distutils
I am now in the Chicago O'Hare airport. I actually found an electrical
outlet next to a seat. What a rare thing in this airport. I usually
end up sitting on the floor, often under a drinking fountain or pay
phone.
I have been working on packaging up segy-py a bit better while on the plane, but first want to write a bit about some thoughts on python segy driver files. I keep discovering little design problems that show off the power of python. I realized one today. I was thinking about how I have to create a large python file full of dictionaries for each flavor of segy. If this were C/C++ this would definitely be the case or I could write a whole bunch of wrapper functions to go into other definitions for functionality. In python, it turns out that it should be really easy to pull parts of functionality from different config files. The example that I was thinking of was with a segy flavor that is almost segy rev1.
PACKAGING
In order for people to be able to use my segy-py package, it needs to work as a solid python package. As such, I have been trying to use the standard distutils. This started off pretty quickly, but then I ran into a few things that I figured out by looking at other packages. The first one is that I hate having to list out every source file. There ways to do this with GNU Make, but it is complicated for a beginner. With python it turns out to be simple, but I am not sure the difference between the two. I started out with something like this inside of the setup call in setup.py:
The next question is should I be putting my pure python sources under a subdirectory or can I leave them all in the top level right now? Will everything at the top level cause install problems? Looking in the fink directory /sw/lib/python2.4/site-packages I see a mess. There are directories, python files, .so compiled shared libraries, pth files (what are these?), and other junk. I will have to get back to this one later.
Then on to a Makefile. I almost always end up with a Makefile for everything I do. People who work on projects with me know that I love simple GNU makefiles and bash scripts. Bash is falling quickly by the wayside with python, but I have a makefile for this project. Take a quick look at part of my makefile:
Why am I bringing up make other than the fact that I think make is cool? Well, the default setup.py file I had does not put the Makefile in the tarball. Doh! # Files to also put in the distribution (sdist or bdist)
# Active files include Makefile
# Documentation files include COPYRIGHT ChangeLog README TODO Then I did a make sdist (which is just ./setup.py sdist --formats=bztar) and I now have a MANIFEST that shows:
Now what exactly is __init__.py supposed to do? None of them seem to really have any info in them. I do not yet see a purpose.
DB WORK BEGINNING...
It is now time to start working on the database side of the house. I want these drivers to be able to create and/or flush out an sql database. I would prefer the design to be generic enough that it will work with any database that I may want to interface with ( for example: sqlite, mysql, postgres, oracle, and so on). I am thinking that the simplest way (but not the best for absolute performance) is to have the drivers be able to return SQL strings. So for a trace, I can first get a CREATE TABLE string. Then I can ask for an INSERT INTO string. Then from outside the driver I can work with the database how I see fit.
The other key idea I have is to either take a list of field that will be used or dynamically generate a string that inspects the dictionary in the driver to see what table fields should be called. The later may cause serious trouble when trying to build a large database of SEGY info between instruments.
Soon I will be running into another problem. The contractor SEGY from the 70's that is available on the USGS walrus database does not have navigation in the SEGY. That comes as a separate file. Somehow, I will have to shoehorn that into the driver for that older SEGY Rev 0 data. Joy. Right now, I have a parser called sins_nav_data.py that handles the navigation data. What does SINS stand for?
That is enough writing for now. If you made it this far into what I wrote, I am impressed. Time to get back to coding this up. When I am back in San Diego, I will have to transition from these couple days of coding back to writing up the data for my thesis.
I have been working on packaging up segy-py a bit better while on the plane, but first want to write a bit about some thoughts on python segy driver files. I keep discovering little design problems that show off the power of python. I realized one today. I was thinking about how I have to create a large python file full of dictionaries for each flavor of segy. If this were C/C++ this would definitely be the case or I could write a whole bunch of wrapper functions to go into other definitions for functionality. In python, it turns out that it should be really easy to pull parts of functionality from different config files. The example that I was thinking of was with a segy flavor that is almost segy rev1.
#!/usr/bin/env pythonRunning this as './segy_tweak.py' gives results like this:
import segy_rev1 print segy_rev1.textFileHeaderEntries['LINE']
from segy_rev1 import textFileHeaderEntries
print textFileHeaderEntries['LINE'] textFileHeaderEntries['LINE'] = [3,5] print textFileHeaderEntries['LINE']
print segy_rev1.textFileHeaderEntries['LINE']
[2, 5] [2, 5] [3, 5] [3, 5]This show that there is one problem or drawback to this approach. The original module data is altered. If you had two different segy files open with different drivers for each, they could step on each other, right? I did a quick check and this is a global problem (or feature if it helps you for other code). So a better way to solve this trickiness is to do a deep copy of the data.
#!/usr/bin/env pythonWhich shows that the deep copy does what we want in this case:
import segy_rev1 print segy_rev1.textFileHeaderEntries['LINE']
import copy textFileHeaderEntries = copy.deepcopy(segy_rev1.textFileHeaderEntries)
textFileHeaderEntries['LINE'] = [3,5] print textFileHeaderEntries['LINE']
print segy_rev1.textFileHeaderEntries['LINE']
./segy_tweaked3.py [2, 5] [3, 5] [2, 5] <-- The original value is not changed. Perfect!
PACKAGING
In order for people to be able to use my segy-py package, it needs to work as a solid python package. As such, I have been trying to use the standard distutils. This started off pretty quickly, but then I ran into a few things that I figured out by looking at other packages. The first one is that I hate having to list out every source file. There ways to do this with GNU Make, but it is complicated for a beginner. With python it turns out to be simple, but I am not sure the difference between the two. I started out with something like this inside of the setup call in setup.py:
py_modules = ['segy','ebcdic','segy_sql','ibm'],There I have listed out each python file that I want to end up in the distributed tarball. But what I really want is every python file included. I found an example that does this:
packages=['']So what really is the difference between a module and a package? I bet I will discover that when I get around to trying to install the stuff with setup.py.
The next question is should I be putting my pure python sources under a subdirectory or can I leave them all in the top level right now? Will everything at the top level cause install problems? Looking in the fink directory /sw/lib/python2.4/site-packages I see a mess. There are directories, python files, .so compiled shared libraries, pth files (what are these?), and other junk. I will have to get back to this one later.
Then on to a Makefile. I almost always end up with a Makefile for everything I do. People who work on projects with me know that I love simple GNU makefiles and bash scripts. Bash is falling quickly by the wayside with python, but I have a makefile for this project. Take a quick look at part of my makefile:
PYTHON_FILES:=${wildcard *.py} PYTHON_MODS:=${PYTHON_FILES:.py=} PYTHON_MOD_DOCS:=${PYTHON_FILES:.py=.html} docs: @echo "${PYTHON_MODS}" pydoc -w ${PYTHON_MODS} code2html Makefile > Makefile-segy-py.html BASE_URL:=http://schwehr.org/software/segy-py BASE_SCP:=schwehr.org:www/software/segy-py docs-install: docs scp ${PYTHON_MOD_DOCS} Makefile-segy-py.html ${BASE_SCP}/docs open ${BASE_URL}/docs help: open ${BASE_URL}/docs check: test pychecker *.py grep -n FIX *.py test: ./segy.py -v clean: rm -f *.pyc *.html *~It really does not do very much, but it helps me remember common tasks. For example, testing the code and uploading the web documentation to my server becomes as easy as this:
# dist is build by "python setup.py sdist" # and MANIFEST gets built in the process based on MANIFEST.in dist-clean: clean rm -rf dist *.db rm -rf MANIFEST
# python packaging sdist: dist-clean ./setup.py sdist --formats=bztar sdist-install: sdist scp dist/* ${BASE_SCP}/
make check && make test && make docs-installWell, that does not work right now as I often have tests that fail, but you get the idea.
Why am I bringing up make other than the fact that I think make is cool? Well, the default setup.py file I had does not put the Makefile in the tarball. Doh! # Files to also put in the distribution (sdist or bdist)
# Active files include Makefile
# Documentation files include COPYRIGHT ChangeLog README TODO Then I did a make sdist (which is just ./setup.py sdist --formats=bztar) and I now have a MANIFEST that shows:
COPYRIGHT ChangeLog Makefile README TODO ebcdic.py ibm.py segy.py segy_rev1.py segy_sql.py setup.pyAnd looking at the tar that is produced, everything looks great:
tar tfj dist/segy-py-0.3.tar.bz2Hopefully that will help others get an even quicker jump start on setup.py files than I got (and I started just a day or two ago).
segy-py-0.3/ segy-py-0.3/ChangeLog segy-py-0.3/COPYRIGHT segy-py-0.3/ebcdic.py segy-py-0.3/ibm.py segy-py-0.3/Makefile segy-py-0.3/PKG-INFO segy-py-0.3/README segy-py-0.3/segy.py segy-py-0.3/segy_rev1.py segy-py-0.3/segy_sql.py segy-py-0.3/setup.py segy-py-0.3/TODO
Now what exactly is __init__.py supposed to do? None of them seem to really have any info in them. I do not yet see a purpose.
DB WORK BEGINNING...
It is now time to start working on the database side of the house. I want these drivers to be able to create and/or flush out an sql database. I would prefer the design to be generic enough that it will work with any database that I may want to interface with ( for example: sqlite, mysql, postgres, oracle, and so on). I am thinking that the simplest way (but not the best for absolute performance) is to have the drivers be able to return SQL strings. So for a trace, I can first get a CREATE TABLE string. Then I can ask for an INSERT INTO string. Then from outside the driver I can work with the database how I see fit.
The other key idea I have is to either take a list of field that will be used or dynamically generate a string that inspects the dictionary in the driver to see what table fields should be called. The later may cause serious trouble when trying to build a large database of SEGY info between instruments.
Soon I will be running into another problem. The contractor SEGY from the 70's that is available on the USGS walrus database does not have navigation in the SEGY. That comes as a separate file. Somehow, I will have to shoehorn that into the driver for that older SEGY Rev 0 data. Joy. Right now, I have a parser called sins_nav_data.py that handles the navigation data. What does SINS stand for?
That is enough writing for now. If you made it this far into what I wrote, I am impressed. Time to get back to coding this up. When I am back in San Diego, I will have to transition from these couple days of coding back to writing up the data for my thesis.
05.30.2005 15:00
misc stuff
Disk
recovery from crashes. The inside scoup by Toms Hardware.
WiFi security.
LinuxISOTorrent
iCluster at SIO. Many screen Mac G5 systems.
WiFi security.
LinuxISOTorrent
iCluster at SIO. Many screen Mac G5 systems.
05.30.2005 09:22
pysqlite, python CAD, more segy-py
pysqlite
2.0.3 was just released. I need to soon switch from version 1.0!
Python CAD - no idea about his thing.
fish is a user friendly command line shell for UNIX-like operating systems such as Linux. Sounds interesting.
BRL CAD looks like an interesting CSG CAD system
I have been working on the loadable driver support for segy-py. It is getting there. I think I need to start using a real python IDE. Also, it is tough to debug problems when unit tests do not produce stack traces. The segy-py API is likely to drift a bit.
FreeAntennas.com for wifi antennas.
Python CAD - no idea about his thing.
fish is a user friendly command line shell for UNIX-like operating systems such as Linux. Sounds interesting.
BRL CAD looks like an interesting CSG CAD system
I have been working on the loadable driver support for segy-py. It is getting there. I think I need to start using a real python IDE. Also, it is tough to debug problems when unit tests do not produce stack traces. The segy-py API is likely to drift a bit.
FreeAntennas.com for wifi antennas.
05.29.2005 09:25
first release of segy-py
I have just put together an actual release of segy-py. This is hardly
even alpha code at this point. Rougher than rough. But now it has a
python setup and on the web docs that get automatically updated via a
Makefile. I am still learning how to make a proper setup.py, but it
seems to work.
BTW, I was reading the MarsRoverBlog for a while, but I can not deal with the Martian alien civilization junk. Goodbye.
segy-py
BTW, I was reading the MarsRoverBlog for a while, but I can not deal with the Martian alien civilization junk. Goodbye.
05.28.2005 12:05
python dynamic module loading
I have been wanting to implement a system for dynamically loading of
driver information. I want to have a set of files where each one
contains the python information needed to load SEGY files for that
particular vender version. I would then have one for standard segy
rev1 and possibly one for rev 0. After that, there would be drivers
for xstar/Edgetech and other venders. These files would be a function
to help ID the file followed by custom readers and lookup tables for
what fields are where in the file. Today, I started figuring out how
to load in these files. I was thinking that I would want to load the
module into a "driver" namespace. Doing "import segy_xstar" is not
very flexible. I want to do something like "drv =
load_driver("xstar")" or "drv = load_driver("rev1")".
Turns out that this is not to hard to do! Here is my first test
example using the imp module.
First the file foo.py that will be loaded:
findsounds.com
First the file foo.py that will be loaded:
bar = 'this is in foo'Here is main.py:
#!/usr/bin/env python """ Test out module loading """Then to run it:
import imp
file = open('foo.py','r') modFoo = imp.load_module('chirp',file,'foo.py',('.py','r',imp.PY_SOURCE))
print modFoo.bar
chmod +x main.py ./main.py
this is in foo
findsounds.com
05.28.2005 09:28
BitTorrent
Perhaps BitTorrent, eDonkey, or some other P2P mechanism is the real
way that we should be distributing marine science data. The files are
huge and it is mandatory that we have a national archive, but it new
surveys were distributed via P2P, people could really get at this
stuff quickly.
Python and Doxygen!
Python and Doxygen!
05.26.2005 10:02
segy python sqlite simple example
This is an example of how to quickly create an sqlite database using
segy python and the sqlite module. This is nothing fancy and it is
missing 99.9% of all the information available, but it is small and
thus easier to understand (I hope). First the code. This is the
previous example, but with a little extra sqlite code.
Constantin Cranganu from Brooklyn College had an interesting over pressure poster this morning.
The Seismic Glosary is helping me get through the Sumatra talks.
CIA Factbook - Just a reminder that it is still with us.
OSS on OSX - this article covers fink too.
Windows guys gives up on MS Windows - An amusing read about someone who has had it with Windows.
#!/usr/bin/env python from segy import Segy import sqliteOnce this is run, you will now have a little sqlite database in the same directory. Here is a simple query to look at a range of traces.
dbFilename = 'bpsegy.db' cx = sqlite.connect(dbFilename) cu = cx.cursor()
# traceCount is the trace offset in a file createStr = "CREATE TABLE traces" createStr += " ( traceCount INTEGER, lat REAL, long REAL );" cu.execute(createStr) cx.commit()
xstar = Segy('bpsio01L00.xstar')
for i in range(1,xstar.getNumberOfTraces()): traceHdr = xstar.getTraceHdr(i)
x = traceHdr.getHeader('X')/3600. y = traceHdr.getHeader('Y')/3600.
insertStr = "INSERT INTO traces" insertStr += " (traceCount, lat,long)" insertStr += " VALUES" insertStr += " (" + str(i) insertStr += " ,"+str(x) insertStr += " ,"+str(y) insertStr += " );" cu.execute(insertStr)
# Flush this to the db... cx.commit()
sqlite bpsegy.db "select * from traces where traceCount>40 and \ traceCount<50;" | tr '|' ' 'That is it for now.
41 -120.101111111 34.4061111111 42 -120.101111111 34.4061111111 43 -120.101111111 34.4061111111 44 -120.101111111 34.4061111111 45 -120.101111111 34.4061111111 46 -120.101111111 34.4061111111 47 -120.101388889 34.4061111111 48 -120.101388889 34.4061111111 49 -120.101388889 34.4061111111
Constantin Cranganu from Brooklyn College had an interesting over pressure poster this morning.
The Seismic Glosary is helping me get through the Sumatra talks.
CIA Factbook - Just a reminder that it is still with us.
OSS on OSX - this article covers fink too.
Windows guys gives up on MS Windows - An amusing read about someone who has had it with Windows.
05.26.2005 09:16
NAMSS and segy python example
NOTE: For the USGS maps, you will need a working SVG plugin. Firefox
has one built in, while you will have to install one from Adobe for
Safari.
I talked to Hart and Childs this morning about the National Archive of Marine Seismic Surveys or NAMSS. What really caught my eye on their poster was the Santa Barbara Basin!
From the uswest page there are a number of marked surveys in the Santa Barbara area. The red dots are not yet available, but the green are available!
Now back to segy python! I now have a working shot location dumper. I was caught off guard by the X,Y not changing in the first few traces, but I should have remembered that is the case. The traces always have the last available X,Y such that it looks like traces do not move for a while unit the next GPS data. I need to make some sort of interpolation function for this. Here is my simple trace, shotpoint, x and y dumper program.
Then the code loops over all the traces. The range must have a "1," in it as there is no trace 0 in the SEGY specification. If you ask for trace 0, you will get garbage.
For each loop, the getTraceHdr(i), pulls out another mmap'ed reference to a particular trace. From the trace reference, I can then ask for the X,Y, and shotpoint fields. I know from prior knowledge (but you could look at the header fields too) that these X,Y pairs are in seconds of arc. To get decimal lat/long, I divide by 3600. Then I print it out. That is it! Here is the beginning and end of the output:
Other news: New Method For Imaging Dec. 26 Indian Ocean Earthquake Yields Unprecedented Results. Mentions Scripps and IGPP
I talked to Hart and Childs this morning about the National Archive of Marine Seismic Surveys or NAMSS. What really caught my eye on their poster was the Santa Barbara Basin!
From the uswest page there are a number of marked surveys in the Santa Barbara area. The red dots are not yet available, but the green are available!
Now back to segy python! I now have a working shot location dumper. I was caught off guard by the X,Y not changing in the first few traces, but I should have remembered that is the case. The traces always have the last available X,Y such that it looks like traces do not move for a while unit the next GPS data. I need to make some sort of interpolation function for this. Here is my simple trace, shotpoint, x and y dumper program.
#!/usr/bin/env python from segy import Segy xstar = Segy('bpsio01L00.xstar')Creating a Segy option with a filename loads up the datafile as a very efficient memory map. This way the data from the file only comes into memory when needed. The hope if that if you are only reading, say, one trace, the file header and that trace are paged in, but thing else.
for i in range(1,xstar.getNumberOfTraces()): traceHdr = xstar.getTraceHdr(i)
shot = traceHdr.getHeader('Shotpoint') x = traceHdr.getHeader('X')/3600. y = traceHdr.getHeader('Y')/3600. print shot,": ",x,y
Then the code loops over all the traces. The range must have a "1," in it as there is no trace 0 in the SEGY specification. If you ask for trace 0, you will get garbage.
For each loop, the getTraceHdr(i), pulls out another mmap'ed reference to a particular trace. From the trace reference, I can then ask for the X,Y, and shotpoint fields. I know from prior knowledge (but you could look at the header fields too) that these X,Y pairs are in seconds of arc. To get decimal lat/long, I divide by 3600. Then I print it out. That is it! Here is the beginning and end of the output:
./segy_sql.py
1 13961172 : -120.101111111 34.4061111111 2 13961172 : -120.101111111 34.4061111111 3 13961172 : -120.101111111 34.4061111111 4 13961172 : -120.101111111 34.4061111111 5 13961172 : -120.101111111 34.4061111111 6 13961172 : -120.101111111 34.4061111111 7 13961172 : -120.101111111 34.4061111111 8 13961172 : -120.101111111 34.4061111111 9 13961172 : -120.101111111 34.4061111111 10 13961172 : -120.101111111 34.4061111111 ... 13540 13961172 : -120.1275 34.4308333333 13541 13961172 : -120.1275 34.4308333333 13542 13961172 : -120.1275 34.4308333333 13543 13961172 : -120.1275 34.4308333333 13544 13961172 : -120.1275 34.4308333333 13545 13961172 : -120.1275 34.4308333333 13546 13961172 : -120.1275 34.4308333333 13547 13961172 : -120.1275 34.4308333333 13548 13961172 : -120.1275 34.4308333333 13549 13961172 : -120.1275 34.4308333333
Other news: New Method For Imaging Dec. 26 Indian Ocean Earthquake Yields Unprecedented Results. Mentions Scripps and IGPP
05.26.2005 06:10
SIO, Audio books
netlibrary is the LA
libraries free audio book service.
NASA's Rovers Continue Martian Missions
Researchers Discover Underwater Volcano -- Unique Hydrothermal Community Of Hundreds Of Eels Uncovered - Quotes by Hubert!
Lots of programming tutorials - SQL, Cobol, Assembly, XML
NASA's Rovers Continue Martian Missions
Researchers Discover Underwater Volcano -- Unique Hydrothermal Community Of Hundreds Of Eels Uncovered - Quotes by Hubert!
Lots of programming tutorials - SQL, Cobol, Assembly, XML
05.25.2005 13:58
Segy python works. Woohoo
UPDATE 17:00 - Turns out that the API below is already out of date, but I just made my first plot of a trace dumped to a sample number/value pair. Looks very much like I expected!
Talking to quite a number of people about seismic data yesterday got me motivated to really go for it with my python segy library. The task really has been mostly an exercise in making lots of lookup tables with dictionaries. Just a few minutes ago, I got my first correct lat/long pair out of a trace from an Edgetech Xstar segy file. This is extra exciting since the xstar format does not follow the SEGY standards. In fact, seismic unix (aka su or cwp-su) is unable to read xstar files. Right now, we first run them through sioseis to write out a better segy file. My hope is to have code that can easily be adapted to each vender's interpretation of the SEGY standards. Clearly people treat the SEGY documentation as suggestions. Being that I intend this to be an open source utility, here is the source. Note that this is just my test code. It is FUGLY.
segy-1.7.py.htmlHere is a little bit of pseudo code that shows how I can read from the first file/trace of the Santa Barbara data. I open up the file by creating a Segy class object and from there, I create a SegyTrace object. One problem right now is that it is possible to close the mmap from the Segy class while SegyTrace objects are still using the mmap. I do not know what would happen if that happens. Pseudo code:
import Segy xstar = Segy("bpsio01L00.xstar") trace1 = SegyTrace(xstar.data,3600) trace1.printHeader() SweepFreqStart = 1000 RecvGrpElev = 0 SweepLen = 50 OverTravel = 392 GapSize = 235 SrcEDirTenths = 0 LowCutFreq = 0 ... # look! If we divide by 3600, we get degrees! x = trace1.getHeader('X'); print x, '--> ',x/3600. y = trace1.getHeader('Y'); print y, '--> ',y/3600. -432364 --> -120.101111111 123862 --> 34.4061111111Those lat/lon numbers actually look like they are in the right area. Party time! Actually, now I need to be able to seek through the traces. I have not done that yet, hence the hard coded 3600 for the trace start. Then after that, I need to build and sqlite database of the file and trace parameters.
fugal adj : of or relating to or in the style of a musical fugue
fugue n 1: dissociative disorder in which a person forgets who who they are and leaves home to creates a new life; during the fugue there is no memory of the former life; after recovering there is no memory for events during the dissociative state [syn: {psychogenic fugue}] 2: a dreamlike state of altered consciousness that may last for hours or days 3: a musical form consisting of a theme repeated a fifth above or a fourth below its first statement
XML tutorials
05.25.2005 08:16
AGU and more
Yesterday, I talked to lots of interesting people in the OS and PP
groups. I also went to the lunch meeting for the AGU informatics
group. They spent their time on beaurocratic drudgery. Who is going
to host the web site and what is the group to be called. Please.
That is not what I went for!
MySQL cheat sheet - it is not nice to cheet.
Yafray is a raytracer that blender can use.
Verse is some sort of networked visualization/geometry system. May have some viz like aspects to it and may take the idea a lot farther, moving the visualization away from the center. The exact opposite from Viz.
PC Assembly tutorial - Intel 32-bit protected mode programming with NASM. What about PPC? Mac PCC assembly links
Beginners Guide - PowerPC Assembly Language
dumb laws
WiseGeek.com - Clear Answers for Common Questions.
MySQL cheat sheet - it is not nice to cheet.
Yafray is a raytracer that blender can use.
Verse is some sort of networked visualization/geometry system. May have some viz like aspects to it and may take the idea a lot farther, moving the visualization away from the center. The exact opposite from Viz.
PC Assembly tutorial - Intel 32-bit protected mode programming with NASM. What about PPC? Mac PCC assembly links
Beginners Guide - PowerPC Assembly Language
dumb laws
WiseGeek.com - Clear Answers for Common Questions.
05.23.2005 19:41
reading binary files with python
Now that I finished my poster presentation today at AGU, I wanted to
take a quick look at reading segy files with python. I wanted to read
straigt from mmap, but did not know how. Now I do. I have not
started working with byte order, but here is the basics. Byte order
can be done with Module-socket
with the htonl type functions. Or better yet with the struct module
with the "@=<>"
SEG publications: segy rev 0 and reb 1 are on this page.
Here is a little program that I wrote to tell byte order for xsonar. GCC can also tell you byte order, and I really need to write or find a couple liner in python.
First two write some binary data with a simple C++ program:
SEG publications: segy rev 0 and reb 1 are on this page.
All values in the binary File Header and the Trace Header are two's compliment integers, either two bytes or four bytes long. There are no floating-point values defined in the headers.
Trace Data sample values are either two's complement integers or floating-point. This revision adds data sample formats of 8-bit integer and 32-bit IEEE floating-point. Both IBM floating-point and IEEE floating-point values are written in big-endian byte order.
Here is a little program that I wrote to tell byte order for xsonar. GCC can also tell you byte order, and I really need to write or find a couple liner in python.
#include <stdlib.h> #include <unistd.h> #include <stdio.h>Mac OSX is big endian and the same byte order as SEGY which will help performance. But I must make sure that I check out the code on an Intel machine.
enum endian {LITTLE,BIG,UNKNOWN_ENDIAN}; // aka LOW, HIGH
enum endian getHostEndian(void) { int short s=1; char *b = (char *)&s; if (b[0]) return (LITTLE); return (BIG); }
int main (int argc, char *argv[]) { switch (getHostEndian()) { #if 0 case LITTLE: printf("little\n"); break; case BIG: printf("big\n"); break; #else case LITTLE: printf("-DINTEL=1 -DMOTOROLA=0\n"); break; case BIG: printf("-DINTEL=0 -DMOTOROLA=1\n"); break; #endif case UNKNOWN_ENDIAN: printf("ERROR: UNKNOWN endian\n"); exit(EXIT_FAILURE); default: printf("ERROR: unexpected endian\n"); exit(EXIT_FAILURE); }; return(EXIT_SUCCESS); }
/* NOTES - What machines return, what xsonar returns * * Darwin/Mac OSX/PPC - big * IRIX/SGI/Mips - big, MOTOROLA * Linux/x86 - little, INTEL */
First two write some binary data with a simple C++ program:
#include <unistd.h> #include <cstdio> #include <fcntl.h> #include <cstdlib>
using namespace std;
int main (int argc,char *argv[]) { { int fd = open("int",O_WRONLY|O_CREAT,0644); int32_t _int32=6; write(fd,&_int32,sizeof(_int32)); close(fd); } { int fd = open("int_neg",O_WRONLY|O_CREAT,0644); int32_t _int32=-6; write(fd,&_int32,sizeof(_int32)); close(fd); } { int fd = open("float",O_WRONLY|O_CREAT,0644); float d=6.77; write(fd,&d,sizeof(float)); close(fd); } { int fd = open("float_neg",O_WRONLY|O_CREAT,0644); float d=6.78; write(fd,&d,sizeof(float)); close(fd); } return (EXIT_SUCCESS); }
#!/usr/bin/env pythonWhen I run the python code, it reads these values correctly:
import mmap import os import struct
def readInt(filename): file = open(filename,"r+") size = os.path.getsize(filename) data = mmap.mmap(file.fileno(),size) return struct.unpack("i",data[:4])[0]
def readFloat(filename): file = open(filename,"r+") size = os.path.getsize(filename) data = mmap.mmap(file.fileno(),size) return struct.unpack("f",data[:4])[0]
if __name__ == "__main__": print "testing..." x = readInt("int"); print x x = readInt("int_neg"); print x x = readFloat("float"); print x x = readFloat("float_neg"); print x
./tryMmapStruct.py
testing... 6 -6 6.76999998093 6.78000020981
05.23.2005 08:27
Nanoblogger comment system
I have not tried this, but comments for
nanoblogger
My Boring Ass Life - Silent Bob Speaks in his blog.
My Boring Ass Life - Silent Bob Speaks in his blog.
05.22.2005 19:26
In NOLA
I'm here for the AGU/SEG conference. Please come by my poster
tomorrow afternoon and Leah Hogarth has hers tomorrow morning.
I am having trouble finding ANY Two Dog beer.
C++ Exceptions scare me still. I use exceptions in python, but never really tried in C++.
Cruise on over to google labs if you have not already. The only thing I would warn against is their desktop search tool and web accelarator. Large chances of google caching personal information.
RentACoder - maybe a way to earn cash.
MySQL tips
How Effective is Your Test Automation?
Using Performance Test Tools for High Volume Automated Testing
I am having trouble finding ANY Two Dog beer.
C++ Exceptions scare me still. I use exceptions in python, but never really tried in C++.
Cruise on over to google labs if you have not already. The only thing I would warn against is their desktop search tool and web accelarator. Large chances of google caching personal information.
RentACoder - maybe a way to earn cash.
MySQL tips
How Effective is Your Test Automation?
Using Performance Test Tools for High Volume Automated Testing
05.21.2005 21:57
Off to AGU New Orleans
Monday starts AGU/SEG.
Please stop by the Monday afternoon GP poster session and check out my magnetic susceptibility poster.
For you folks still using Matlab: Matlab contest winners
Please stop by the Monday afternoon GP poster session and check out my magnetic susceptibility poster.
For you folks still using Matlab: Matlab contest winners
05.20.2005 21:58
New version of global paleomag database
The Global paleomagnetic database Bersion 4.6,
February 2005 is out. I do not use this myself right now, but is
cool to have available. Only bummer is that it is avaiable as an
access database... that would mean that I would have to go use a
Windows XP machine. Or is there something in the open source world
that will read MS Access database files? There looks to be some stuff
on freshmeat.net that can read MS
Access databases. The one that seems to be the most likely for
success is: mdbtools.
The mailing list has a number of emails from this year and 0.6pre1 is
from last summer. This looks to do the trick, but I have not tried
it.
Royalty free stock photos
Royalty free stock photos
05.20.2005 13:56
Not My Workstation OS - Irix
In the past, I was a hard core IRIX/SGI fan. Those were the days when
NASA put a $50K machine on my desk (Octane MXI - Dual 200MHz R10K). I
was just exploring SGI's OpenInventor 2.x and the SGI user interface
was way beyond Mac OS (was it 7 or 8 in 1996/7). There was the
freeware CD which was so-so but worked, I build gcc to cross compile
with 68K VxWorks, and we could use the new MipsPro compilers to do
parallel processing for our stereo code. Life was good. Then I
realized that I was using PC, Mac, VxWorks cage, Linux, SGI, and Sun
all in one day. Yikes. Now I am a happy user of Mac OSX and Linux.
The last stand for me with Irix is the 21 foot wide panoram screen in
the viz center. Otherwise, Irix was a fantastic OS. It still
has many advancements that are yet to be seen on other OSes. Someday
I should rewrite button fly for any OS so then I can have flashbacks
to my early days with Irix. I will never forget learning computer
graphics at Stanford on those "super fast" R3K machines in the
basement of Sweet Hall and the old computer science building.
My Workstation OS: Irix by Robert Mertling-Blake who apparently is a huge Irix fan. More power to him.
My Workstation OS: Irix by Robert Mertling-Blake who apparently is a huge Irix fan. More power to him.
05.20.2005 09:15
Wave farm in Portugal
I thought there were already some wave farms going, like in Scotland,
but this is pretty cool.
Portugal to get world's first commercial wave farm
And some news about the people who killed the Giant Sea Bass in La Jolla. Finally San Diego does something smart. Fishing in these areas hurts the local protected populations and hurts the tourist industry that this area counts on. Now for making the Children's Cove back into a protected seal area! The seals are a huge tourist attraction and there are plenty of other beaches for people to go to!
Apple battery recall:
Veusz - yet another python plotting package.
We really need a Mac version of the python win32 video capture package. Working with the quicktime api directly is very frustrating.
Terminal from OSX 10.4 DVD - Nice. Now I need time to install 10.4.
PythonCallenge! - Also need more time for this.
Portugal to get world's first commercial wave farm
And some news about the people who killed the Giant Sea Bass in La Jolla. Finally San Diego does something smart. Fishing in these areas hurts the local protected populations and hurts the tourist industry that this area counts on. Now for making the Children's Cove back into a protected seal area! The seals are a huge tourist attraction and there are plenty of other beaches for people to go to!
FOR IMMEDIATE RELEASE: May 19, 2005 Contact: Maria Velasquez, Communications Director: (619) 235-5725 (pager & voicemail) mvelasquez@sandiego.gov
CRIMINAL CHARGES FILED AGAINST SEA BASS POACHERS Complaint filed by San Diego City Attorney's Office
San Diego, CA: The Office of the San Diego City Attorney has filed criminal charges against three San Diego men involved in the April 24, 2005, killing of a protected Giant Sea Bass taken from the La Jolla Marine Conservation Area. Omid Adhami, 34 years old, has previously admitted to using a spear gun to kill the protected fish. Nima Hodaji, 26 years old, was diving with Adhami at the time the fish was killed. Navid Adibi, 22 years old, was operating the boat that took the two divers into the Conservation Area.
The Criminal Complaint filed in San Diego Superior Court alleges that all three men assisted in loading the 171 pound protected Giant Sea Bass onto the boat in the Marine Conservation Area off of La Jolla. Two counts of misdemeanor conspiracy have been filed against all three defendants who face up to one year in jail and a $10,000 fine on each conspiracy charge. According to Deputy City Attorney Kathyrn Lange of the City Attorney's Consumer and Environmental Protection Unit, Adhami faces a total four misdemeanor counts (three years in jail and $22,000 in fines), Hodaji and Adibi each face three misdemeanors counts (two and one-half years in jail and $21,000 in fines.) Arraignment of the three defendants will take place on June 2, 2005, in Department 1 of the San Diego Superior Court.
Giant Sea Bass are described as docile, curious, and slow moving fish. They are a protected species off of the California Coast. The Giant Sea Bass killed on April 24, 2005, was a reproductive female between 15 and 20 years old. The suspicious activity of the three men was first noticed by the City of San Diego Lifeguards located at the La Jolla Cove. The case is under investigation by the California Department of Fish and Game.
Apple battery recall:
The recalled batteries include those with model numbers A1061, A1078, and A1079 and serial numbers that begin with HQ441 through HQ507 and 3X446 through 3X510
Veusz - yet another python plotting package.
We really need a Mac version of the python win32 video capture package. Working with the quicktime api directly is very frustrating.
Terminal from OSX 10.4 DVD - Nice. Now I need time to install 10.4.
PythonCallenge! - Also need more time for this.
05.20.2005 05:16
more on the butterflies
I have not seen these butterflies since my last post on there, but saw
an article on the painted lady butterflies: Butterfly
Migration Could Be Largest Known
Earthquake possibility in northern LA: Los Angeles 'Big Squeeze' Continues, Straining Earthquake Faults
A python unit conversion program that includes magnetic units: gonvert. I need to test the magnetics units. Here are some of the units:
Earthquake possibility in northern LA: Los Angeles 'Big Squeeze' Continues, Straining Earthquake Faults
A python unit conversion program that includes magnetic units: gonvert. I need to test the magnetics units. Here are some of the units:
Magnetic Field strength : ampere per meter oersted Magnetic Flux : microweber milliweber unit pole (electro magnetic unit) weber Magnetic Flux Density : gauss tesla weber per square meter
05.19.2005 08:09
New multilanguage SQLite interface API
SQLitepp
"is a multilanguage object oriented wrapper to the sqlite library. It
supports self-updatable records and a simple SQL-based query method."
This was just announced
on freshmeat.net
Python on the Playtation?
Posted by Kurt | Permalink
Python on the Playtation?
Posted by Kurt | Permalink
05.18.2005 18:34
php
I did not know until just now that you can use php as a command line
scripting language and it is available on on Mac OSX by default:
Tom's Networking on How To Crack WEP - Part 2: Performing the Crack
$ type -a phpTo get you started, here is a very simple program to get started with: test.phps
php is /usr/bin/php
$ php -v
PHP 4.3.10 (cli) (built: Jan 23 2005 21:23:17) Copyright (c) 1997-2004 The PHP Group Zend Engine v1.3.0, Copyright (c) 1998-2004 Zend Technologies
type -a php5
php5 is /sw/bin/php5 (this is fink)
$ php5 -v
PHP 5.0.4 (cli) (built: May 15 2005 12:55:54) Copyright (c) 1997-2004 The PHP Group Zend Engine v2.0.4-dev, Copyright (c) 1998-2004 Zend Technologies
#!/usr/bin/php <? $str="World"; print "Hello $str\n"; ?>Which you can run like this.
./chmod +x test.phps ./test.phpsAnd you will see something like this
./test.phpsNow, I really have no need to write php5 scripts, but it is nice to know that I can if I want to
Hello World
Tom's Networking on How To Crack WEP - Part 2: Performing the Crack
05.18.2005 07:13
Computer Graphics World - May 2005
Today is May 18th - the 25th anniversary of the big Mt. St. Helens
eruption. At 8:32 Sunday morning,
May 18, 1980, Mount St. Helens erupted.
Computer Graphics World is one of those skinny little trade rags that say they cost but always end up giving you free subscriptions that never end. The May 2005 issue has some interesting things worth mentioning.
First, I have been thinking that there has to be someone doing a survey of new visualization and UI (User Interface) designs being used in computer games. There are like 12000 playstation games to date. That is a huge amount of engineering energy that had to get the look and feel right. How can we harness this for the marine and geo-sciences? So what is the very first text in this months issue? Exactly this topic. The "editor's note" is about Marc Prensky who wrote Digital Game-Based Learning. He goes through the top 10 nuggets. They did not do much for me, but the idea is right!
Then there is an add by 3Dconnexion for the SpacePilot. I have been thinking about a visualization workstation that combined a spacemouse for one hand and a dial for the other. I was thinking there are a lot of untried ideas for a dial:
"nobody-ever-got-for-buying-intel-world" - Those folks are missing out on some great stuff!
For NASA spacecraft, that would be: "nobody ever got fired for using VxWorks"
URL dmg attaching for Mac OSX:
Computer Graphics World is one of those skinny little trade rags that say they cost but always end up giving you free subscriptions that never end. The May 2005 issue has some interesting things worth mentioning.
First, I have been thinking that there has to be someone doing a survey of new visualization and UI (User Interface) designs being used in computer games. There are like 12000 playstation games to date. That is a huge amount of engineering energy that had to get the look and feel right. How can we harness this for the marine and geo-sciences? So what is the very first text in this months issue? Exactly this topic. The "editor's note" is about Marc Prensky who wrote Digital Game-Based Learning. He goes through the top 10 nuggets. They did not do much for me, but the idea is right!
Then there is an add by 3Dconnexion for the SpacePilot. I have been thinking about a visualization workstation that combined a spacemouse for one hand and a dial for the other. I was thinking there are a lot of untried ideas for a dial:
- could dial time in the model - just like my Sony VCR (remember VCR's right?)
- spin along a flight path
- dial in a brightness
- how to navigate a color space?
- spin an attached view, or...
- add yet one more degree of freedom to camera motion with a spacemouse
- and lots more
"nobody-ever-got-for-buying-intel-world" - Those folks are missing out on some great stuff!
For NASA spacecraft, that would be: "nobody ever got fired for using VxWorks"
URL dmg attaching for Mac OSX:
After reading the hdid man (via man hdid) page, I realized that we could mount .DMG images via HTTP under Terminal.app. And even cooler, the segmented images work too. Just reference the http-based DMG via this command: % hdid http://server.company.com/Images/stuff.dmg
...
so the correct command is hdiutil attach URL. works fine for me.
05.18.2005 05:08
Fun with MS Windows
Arg. On our windows XP box we have been having yet more "issues." "ViewMgr" keeps popping up in zone alarm.
ViewMgr info
So why does each program need to have it's own updater that stays running on a system. That seems insane. If every program that one user had on a machine did the same thing, the machine would be clobbered just by the shear number of processes. Then there is another thing driving me crazy. A person was trying to use an GUI based ftp program to upload some stuff to a site. First off, they all want $30-$50 for an ftp program. You have got to be kidding. WinSCP is free and kicks all of their butts. If only the machine she were using allowed sftp... doh. Then, when she runs these ftp programs, norton says:
Many tutorials: http://www.pixel2life.com/
Air travelers stripped bare with X-ray machine
New Collision Looks Imminent For B-15A Iceberg
http://www.projectorcentral.com/video_signals.htm - The Difference Between HDTV, EDTV, and SDTV
ViewMgr info
So why does each program need to have it's own updater that stays running on a system. That seems insane. If every program that one user had on a machine did the same thing, the machine would be clobbered just by the shear number of processes. Then there is another thing driving me crazy. A person was trying to use an GUI based ftp program to upload some stuff to a site. First off, they all want $30-$50 for an ftp program. You have got to be kidding. WinSCP is free and kicks all of their butts. If only the machine she were using allowed sftp... doh. Then, when she runs these ftp programs, norton says:
Default Block Bla Trojan Horse c:\WINNT\system32\ntvdm.exe Port 1042Norton then gives you no further help or explanation. It looks like it is complaining about active ftp. Most users will opt to block this "trojan" and then have ftp not work. Is this the right conclusion? If not, PLEASE, someone set me straight!
Many tutorials: http://www.pixel2life.com/
Air travelers stripped bare with X-ray machine
New Collision Looks Imminent For B-15A Iceberg
http://www.projectorcentral.com/video_signals.htm - The Difference Between HDTV, EDTV, and SDTV
05.17.2005 13:52
Find cheap gas
http://www.ahding.com/cheapgas/ is down probably because of too much traffic, but this is what we all want. Where is that cheap gas? And it uses google maps too.
gasbuddy.com ... any good? Points to:
http://www.sandiegogasprices.com/ - none of these prices are anywhere near us
gasbuddy.com ... any good? Points to:
http://www.sandiegogasprices.com/ - none of these prices are anywhere near us
05.17.2005 11:08
more python, pysqlite 2.0
Wow... not yet, but I can't wait to dig into this soon: pysqlite 2.0.0 [freshmeat]
For me back in the old days, there is this: basic get-started guide for pysqlite 1.x
a pyblosxom setup
python imap email libs part 2
For me back in the old days, there is this: basic get-started guide for pysqlite 1.x
a pyblosxom setup
python imap email libs part 2
05.17.2005 10:07
python
An attempt at Constant types in Python
Work on a pyblosxom manual - pyblosxom - need time to play with this code!
AMD and Intel processors demistified at anandtech
Looks like rock positioning nicely controlling the location of wind generated ripples:
Work on a pyblosxom manual - pyblosxom - need time to play with this code!
AMD and Intel processors demistified at anandtech
Looks like rock positioning nicely controlling the location of wind generated ripples:
05.17.2005 08:15
SQL delete
This was my first chance to use the SQL DELETE command. Up until now,
the only removal I needed to do was to delete the entire
database as I regenerate it in about 30 sections from all of the flat
file data tables. Now my problem is that I am on a deadline and
crafting a way to compose a graph of remeasured data is a bit of a
challenge (to me at least). I do not want to comment out the older
values from the weights table that now how new measurements, but if I
remove them from the table, I can later remove one function call from
python to nuke that remove set of calls.
Here is one of the removes that I would like to do. For all of section 2 of core 3, I want to remove the first drying where the samples were just set out at room temp to dry (step=1). Since then, I dried the samples in the Norris and Jackson ovens at 50 °C for 2.75 days:
Update 2009-Jan-29: The new URL... http://vislab-ccom.unh.edu/~schwehr/Gaviota/bpsio-Aug04/measuring/
2005-May-16-drying photos
By the way, drying in the fume hood was only started the samples drying. Two days in the fume hood or the vacuum chambers only removed a small fraction of the water.
So how to make the query? Here are the schemas for the weights and the ams tables. I need the ams table since weights does not know about sections, cores, and other handy stuff:
Here is one of the removes that I would like to do. For all of section 2 of core 3, I want to remove the first drying where the samples were just set out at room temp to dry (step=1). Since then, I dried the samples in the Norris and Jackson ovens at 50 °C for 2.75 days:
Update 2009-Jan-29: The new URL... http://vislab-ccom.unh.edu/~schwehr/Gaviota/bpsio-Aug04/measuring/
2005-May-16-drying photos
By the way, drying in the fume hood was only started the samples drying. Two days in the fume hood or the vacuum chambers only removed a small fraction of the water.
So how to make the query? Here are the schemas for the weights and the ams tables. I need the ams table since weights does not know about sections, cores, and other handy stuff:
sqlite db ".schema weights"The final delete query:
CREATE TABLE weights (id INTEGER PRIMARY KEY, samplename VARCHAR(40), weight REAL, timeMeasured TIMESTAMP, status VARCHAR(20), operator VARCHAR(20), sticker VARCHAR(15), step INTEGER);
sqlite db ".schema ams"
CREATE TABLE ams (id INTEGER PRIMARY KEY, samplename VARCHAR(40), user VARCHAR(40), datemeasured TIMESTAMP, cruise VARCHAR(20), corenum INTEGER, coretype VARCHAR(1), corehalf VARCHAR(1), section INTEGER, sectionoffset REAL, depth REAL,counts INTEGER, sampleholder REAL ... and so forth
sqlite db "DELETE FROM weights WHERE weights.id IN (SELECT\ weights.id FROM weights,ams WHERE weights.samplename=ams.samplename\ AND corenum=3 AND section=2 AND weights.step=1);"Before I deleted the samples, I can see lots of the ones I want to be rid of. There should only be
sqlite db "select weights.id,weights.samplename,weights.step from\ weights,ams where weights.samplename=ams.samplename AND corenum=3\ and section=2 AND weights.step=1;" | tail -6After the delete call here is what I see. All is well I hope!
310|bp04-3gw-s2-132|1 311|bp04-3gw-s2-135|1 312|bp04-3gw-s2-138|1 313|bp04-3gw-s2-141|1 314|bp04-3gw-s2-144|1 315|bp04-3gw-s2-147|1
sqlite db "select weights.* FROM weights,ams WHERE weights.samplename=ams.samplename AND step=1 AND corenum=3 AND section=2"Now I need to account for the weight of the stickers.
Nothing
sqlite db "select weights.* FROM weights,ams WHERE \ weights.samplename=ams.samplename AND step=1 AND corenum=3" | head -3
316|bp04-3gw-s1-004|9.7838|2005-04-25 10:30:00|wet|schwehr|no-sticker|1 317|bp04-3gw-s1-007|9.8447|2005-04-25 10:30:00|wet|schwehr|no-sticker|1 318|bp04-3gw-s1-010|9.9916|2005-04-25 10:30:00|wet|schwehr|no-sticker|1
sqlite db "select weights.* FROM weights,ams WHERE \ weights.samplename=ams.samplename AND step=2 AND corenum=3" | head -3
561|bp04-3gw-s2-123|5.9082|2005-05-16 09:14:00|dry|schwehr|sticker|2 562|bp04-3gw-s2-126|5.8825|2005-05-16 09:14:00|dry|schwehr|sticker|2 563|bp04-3gw-s2-141|5.5927|2005-05-16 09:14:00|dry|schwehr|sticker|2
05.17.2005 06:18
M. Chan on Mars Marbles
Yesterday, we had an interesting talk at SIO by Marjorie Chan entitled: "Analogs of Utah Marbles to Mars Blueberries." She makes a strong case for the spheroids on Mars being ground water derived concretions, but I am still not convinced and even if it was ground water, it may not have been very much. I still need to do some serious reading before I officially weigh in on this debate.
Wow. Someone from Moscow sucked down half a gig of stuff from my webserver yesterday. Hello to Moscow!
With spam the way it is today, if you actually won a contest, you would never know.
Google maps + tranceroute - give it a while to bring up the map. Especially today
UI testing resources
WASTE Network:
WASTE (waste.sf.net) is software for setting up small, secure, private filesharing networks. WASTE provides chat, instant messaging, and file transfers between connected users, all of which is encrypted. The software is entirely peer to peer so no server is required. WASTE facilitates connections between NATed users with routing through addressable nodes. It works best for transferring files between small groups of people, like a group of friends.
05.16.2005 19:21
PS3!!!
I am sure that if you care at all about the PS3, you have already
scene this, but I'll put it here anyway. Alex and I spent a lot of
time back in 1999/2000 talking about what we could do if we went in
for a PS2 dev kit. I have never seen any of the products we thought
up. Maybe I can get motivated for the PS3. What about a PS3 on the
bridge of every major ship? It would no longer be just a game.
Anandtech on the PS3. Launch set for 2006.
It looks a little skimpy on RAM, but the unit should be a screamer. And with BlueRay disks, it will have a lot of space for data files. Plus a 2.5 inch HD will make the thing nice and flexible. Ah, 3 GigE port, 6 USB ports and WiFi should make integration a snap. Nice.
The real questions are what will be the price of the machine of the machine be, how much are dev kits, and how hard will it be to program for someone not in a game dev shop?
Anandtech on the PS3. Launch set for 2006.
It looks a little skimpy on RAM, but the unit should be a screamer. And with BlueRay disks, it will have a lot of space for data files. Plus a 2.5 inch HD will make the thing nice and flexible. Ah, 3 GigE port, 6 USB ports and WiFi should make integration a snap. Nice.
The real questions are what will be the price of the machine of the machine be, how much are dev kits, and how hard will it be to program for someone not in a game dev shop?
05.16.2005 14:49
Managing many computers
Here is a slashdot article where the comments might actually be useful
in the future: Updating
Free Software in the Enterprise?
Ask macslash: Bacula for mac and windows?
wallykeyster asks: "I'm an IT Director for a small private university in the U.S., and we are largely a Microsoft shop. We pay over $15,000 each year for our Campus Agreement so that we can upgrade the desktop OS to our version of choice, run Office, and have some Client Access Licenses. I would like to move to FOSS solutions, but I'm having trouble finding support for Enterprise management. For example, OpenOffice and Firefox (both of which I use personally) would be easy first steps, but IE is updated automatically via our SUS server (and settings pushed to clients via group policies) and Office updates will be included soon. How are other larger organizations (i.e. more than 200 desktops) dealing with software deployment and updates? Is anyone using Zen with Novell Desktop Linux?"Maybe someday, I will actually look at CSS and see what it really is... Advanced CSS Layouts: Step by Step
Ask macslash: Bacula for mac and windows?
ZEB17 writes "Anybody out there have any experience using Bacula to back up a Mac? How about with Tiger? We've been Retrospect users forever, but we find their upgrade and support policies a bit tough to swallow. Plus we've never been all that impressed with their software. So we started thinking about the open source project Bacula. It is supposed to work well on heterogenous networks with Linux, Windows, and Mac clients. But doing the usual searches reveals little about actual Mac usage. Is it so good that nobody ever posts anything? Or is it so bad that nobody even tries it? Thanks for any info."
05.16.2005 11:07
inside an G4 TiBook
TizGutz web page has
an xray of the guts of a G4 Titanium laptop:
A chapter on the chapter online about the Mach system from the very good book 'Operating System Concepts'
How-To: BroadCatching using RSS + BitTorrent to automatically download TV shows - not for Macs, but you can adapt, right?
amail - anonymous email
A chapter on the chapter online about the Mach system from the very good book 'Operating System Concepts'
How-To: BroadCatching using RSS + BitTorrent to automatically download TV shows - not for Macs, but you can adapt, right?
amail - anonymous email
05.16.2005 06:13
grid resampling in python
Extracting
a low resolution grid from a high resolution grid
GreaseMonkey for Firefox. I still use Safari, but this sounds pretty cool but does it have serious security issues?
Coral Distribution Network - interesting network of web proxies. Have your page cached.
GreaseMonkey for Firefox. I still use Safari, but this sounds pretty cool but does it have serious security issues?
Coral Distribution Network - interesting network of web proxies. Have your page cached.
05.15.2005 13:20
python doctest
For some reason, I just decided that it is time to start adding
doctest examples into my water analysis code. Then I discovered that
I am not really understanding how this is supposed to work. I thought
my array slicing function would be a good starting point. After a
step back, I have a working example of doctest.
PyStripchart looks very handy. Too bad it is not in fink.
Just noticed that there is python code for Artificial Intelligence: A Modern Approach by Peter Norvig. This is in addition to Lisp and Java. I have an older copy of this book on my shelf.
More free web hosting: http://pho2hosting.com/
#!/usr/bin/env pythonWhich I then run by doing:
def subsetList (list, min, max): """ return a subregion of a list of (x,y) pairs based on the x range
>>> subsetList([(1,11),(2,22),(12,1212),(15,1515),(21,2121),(30,3030)], 10,20) [(12, 1212), (15, 1515)] """ newList=[] for item in list: if item[0]>=min and item[0]<=max: newList.append(item) return newList
def _test(): import doctest, foo doctest.testmod(foo)
if __name__ == '__main__': _test()
chmod +x foo.py ./foo.py -vThe key was to add "-v" after the command name to get more output as to what is going on through the tests.
Trying: subsetList([(1,11),(2,22),(12,1212),(15,1515),(21,2121),(30,3030)], 10,20) Expecting: [(12, 1212), (15, 1515)] ok 2 items had no tests: foo foo._test 1 items passed all tests: 1 tests in foo.subsetList 1 tests in 3 items. 1 passed and 0 failed. Test passed.
PyStripchart looks very handy. Too bad it is not in fink.
Just noticed that there is python code for Artificial Intelligence: A Modern Approach by Peter Norvig. This is in addition to Lisp and Java. I have an older copy of this book on my shelf.
More free web hosting: http://pho2hosting.com/
05.15.2005 10:04
stumbler - viha
I tried to install Viha 0.0.1a and gave stumbler a try. I really just
wanted to see how many wifi access points are in view and what
channels they are using. No go:
Instead, I got MacStumbler to work right off. Turns out that just about everyone in range is using channel 6. Not good! The only thing about macstumbler that I don't like is that it is actively interrogating all channels and devices. kismet/Viha does a passive grab of packets.
I never heard of Forensic Psychology until now:
Linux tips most of which are good for Mac OSX too.
Forumer - Free forums if you want to run an online discussion.
./Stumbler IOConnectMapMemory (ipc/send) invalid destination port (10000003) Abort trapAnd then I run it the top right menu bar goes blank and then comes back. Scary. Time to throw out the framework. Maybe I need to get and build 0.0.2 from source, but I am not going to spend the time to try that.
Instead, I got MacStumbler to work right off. Turns out that just about everyone in range is using channel 6. Not good! The only thing about macstumbler that I don't like is that it is actively interrogating all channels and devices. kismet/Viha does a passive grab of packets.
I never heard of Forensic Psychology until now:
Forensic Psychology is the application of the science and profession of psychology to questions and issues relating to law and the legal system. The word "forensic" comes from the Latin word "forensis," meaning "of the forum," where the law courts of ancient Rome were held. Today forensic refers to the application of scientific principles and practices to the adversary process where specially knowledgeable scientists play a role.File Format List
Linux tips most of which are good for Mac OSX too.
Forumer - Free forums if you want to run an online discussion.
05.14.2005 14:07
Python Numeric exp not faster?
I thought the whole point about Numeric.exp() over the standard python
Math.exp() was that it is faster. I figured I would do a quick check
since I will be doing a lot of fitting to exponential functions. I
got that Numeric is much slower. This is on a 1.5 GHz G4 Ti
laptop with 1GB ram and 10.3.9 using python 2.4.1-1 from fink and
numeric-py24 23.8-11. First the code. It maybe that there is some
internal optimization that I am cancelling out by calling exp this
way.
max=800 print max print Numeric.exp(max) print math.exp(max) gives
#!/usr/bin/env python import math import time import NumericYou just can not get any simpler than that. I do not think that python does any optimization that would just throw out the exp call since it is really a no-op. With C++, I would have to go look at the generated assembly to make sure the call actually happened. Now for the results:
totalMath=0. totalNumeric=0.
for j in range(1,100): start = time.time() for i in range(1,100): x = math.exp(float(i)+.1) end = time.time() totalMath += end-start
start = time.time() for i in range(1,100): x = Numeric.exp(float(i)+.1) end = time.time() totalNumeric += end-start
print " totalMath = ",totalMath print " totalNumeric = ",totalNumeric
./speed.py totalMath = 0.064 totalNumeric = 0.192Ouch max=700 print max print Numeric.exp(max) print math.exp(max)
max=800 print max print Numeric.exp(max) print math.exp(max) gives
700 1.01423205474e+304 1.01423205474e+304 800 inf Traceback (most recent call last): File "./speed.py", line 17, in ? print math.exp(max) OverflowError: math range error
05.14.2005 12:05
ScientificPython fit and fit from gnuplot.py
I was wanting to use fit results from gnuplot in python code to
calculate the residuals to an exponential. I was thinking of doing
something like what I did with ams.py/k15_hext that uses popen2 to
grad the results. That works, but I am thing popen2 is kind of ugly
and it was slow. What does gnuplot.py use to talk to gnuplot?
PyBLD - not that I really know what BLD is, but it does have a data fit capability.
GnuplotBiDir.py:
MayaVI is in fink.
Module Scientific.Functions.LeastSquares
7.6.4 Scientific.Functions.LeastSquares. From BoA - The Bolometer Data Analysis Project. If every project had such documentation!
And another interesting mention: SciGraphica a scientific application for data analysis and technical graphics. It pretends to be a clone of the popular commercial (and expensive) application "Microcal Origin". It fully supplies plotting features for 2D, 3D and polar charts.
Wow. Too many plotting options!
As the others have pointed out, you can call any gnuplot command from Gnuplot.py. The other issue with "fit", however, is how do you get the results of the fit back into Python? This is a problem because Gnuplot.py doesn't have access to gnuplot's standard output. Luckily this is easy--you write the parameters to a "parameters file" (which fortuitously is valid python code) then read them back into python. This is untested but should approximately work: >>> g = Gnuplot.Gnuplot() >>> open('fit.par', 'w').write('m=0\nb=0\n') >>> g('fit m*x+b "data.in" via "fit.par"') >>> g('update "fit.par" "fit2.par"') # saves fitted parameters to fit2.par >>> parameters = {} >>> execfile('fit2.par', globals(), parameters) >>> print parameters {'b': 1.5015, 'm': 1e-30} >>> print parameters['m'], parameters['b'] 1e-30 1.5015 In principle you could also parse "fit.log" to get the gory details about the fit (standard errors, correlation matrix, etc). Hope this helps, Michael -- Michael Haggerty mhagger at alum dott mit.eduFiPy - FiPy: A Finite Volume PDE Solver Using Python
PyBLD - not that I really know what BLD is, but it does have a data fit capability.
GnuplotBiDir.py:
Sometimes it is useful to get variables from a gnuplot session steered from python, e.g. to extract parameters from a fit. GnuplotBiDir.py is a test of bidirectional interfacing with gnuplot. For this you need a really recent gnuplot (from CVS) . The basic usage is from GnuplotBiDir import Gnuplot gp=Gnuplot() gp("a=10") print "var=",gp.getvar("a") Remark: for an excellent python interface to gnuplot see gnuplot-py and the interactive add-ons to this from IPython.Numeric And Scientific packages [python.org]
MayaVI is in fink.
MayaVi is a free, easy to use scientific data visualizer. It is written in Python and uses the amazing Visualization Toolkit (VTK) for the graphics. It provides a GUI written using Tkinter.ScientificPython does nonlinear least squares fits.
Module Scientific.Functions.LeastSquares
7.6.4 Scientific.Functions.LeastSquares. From BoA - The Bolometer Data Analysis Project. If every project had such documentation!
>>> from Scientific.Functions.LeastSquares import * >>> from Numeric import exp >>> def f(param, t): ... return param[0]*exp(-param[1]/t) ... >>> data = [(100, 4.999e-8),(200, 5.307e+2), (300, 1.289e+6),(400, 6.559e+7)] >>> print leastSquaresFit(f, (1e13,4700), data) ([8641551709749.7666, 4715.4677901570467], 1080.2526437958597)I tried the above and it seems to work! I will probably be skipping gnuplot for fitting from now on. It is the same algorithm between the two. We will see if I take to this or not!
And another interesting mention: SciGraphica a scientific application for data analysis and technical graphics. It pretends to be a clone of the popular commercial (and expensive) application "Microcal Origin". It fully supplies plotting features for 2D, 3D and polar charts.
Wow. Too many plotting options!
05.13.2005 19:12
TOR
TOR = Tor: An anonymous Internet
communication system. Uses "onion routers."
Is your port 9100 open? Look out. I wonder if we could make them run grid
:)
An Opportunity rover wheel that is stuck:
Is your port 9100 open? Look out. I wonder if we could make them run grid
:)
An Opportunity rover wheel that is stuck:
05.13.2005 06:31
Weight % water, Mail.app ssl, bash cheat sheet
Yesterday, I finally measured the weight of the "dry" samples. After
talking to a few people, I realize that there are better ways to get
this information. I should have taken samples straight from the oven
at 50 degrees C and put them quickly into a vacuum desiccator.
Instead, my samples were in atmosphere for up to 2 hours before being
weighed. I found this document describing procedures from the Puget
Sound project.
Google dry weight procedure for marine sediments
I do not think the keychain stuff I did yesterday for Mail.app on Mac OSX worked. Maybe I need to logout/log back in? I need to dig in the keychain and make sure the cert is actually in there. Some relevant links:
And yet more Bash stuff: Bash Programming Cheat Sheet
Human-Computer Interaction(HCI)/Viz people that are on my list of people to lookup:
Sedimentary Protocols [pdf]Also, a higher temperature (>= 55° C) will get rid of some volatile organics.
Google dry weight procedure for marine sediments
I do not think the keychain stuff I did yesterday for Mail.app on Mac OSX worked. Maybe I need to logout/log back in? I need to dig in the keychain and make sure the cert is actually in there. Some relevant links:
- Installing Custom Certificate Authorities for OpenSSL by Chris Adams
- Secure IMAPd for OS X 10.3 by Michael Johnson
- http://www.cse.ucsd.edu/users/lbreyer/rootcert/OSX/
- http://www-no.ucsd.edu/documentation/application/macosx/mail/
- http://www-no.ucsd.edu/documentation/application/macosx/mail/ssl/index.html
And yet more Bash stuff: Bash Programming Cheat Sheet
Human-Computer Interaction(HCI)/Viz people that are on my list of people to lookup:
- Tom Furness - uman Interface Technology Lab at UW.
- Alex Pang at UCSC. Research interests: tensor visualization, scientific visualization, collaboration software, uncertainty visualization, virtual reality interfaces.
- Terry Winograd
05.12.2005 09:29
OpenInventor syntax highlighting
I made an offhand post this morning to the coin discussion list and
wow did it get responses! And that was just in an hour. I wanted to
know if it would be possible to emit syntax highlight for open
inventor code,
From the parental news wire service:
Mars rover struggles to escape sand dune
Easily import self-signed SSL certificates on Mac OSX. About time! I am tired of complaints from my mail server!
Bernhard Breinbauer:
Maybe also not what you are looking for, but kwrite/kate from the KDE project have syntax highlighting for vrml files, and also a HTML export function. It's not the direct way, but you get coloured html output :-)
Morton:
Not exactly what you're looking for, but in case it would be useful: Emacs comes with a VRML mode, which also works with Inventor files. As they usually do, the Emacs mode comes with syntax highlighting and indentation. Very useful.
Chris ScharverI figured I might as well throw up an ugly version of my python xcore code for interested parties to look at. This code is UGLY:
GNU enscript is another option. It can pretty-print the file and output to RTF, HTML, or PostScript as needed. It defaults to vrml syntax for .wrl files, but you can force it to use that mode for .iv files too. Very handy! I use it quite a bit on Linux, Mac OS X, and Windows via cygwin.
http://mega.ist.utl.pt/~ic-cg/laboratorio/vrmlTutorials/tutorial/source/tut7.html
http://schwehr.org/xcore/xcore-py-0.1.tar.bz2
From the parental news wire service:
Mars rover struggles to escape sand dune
Easily import self-signed SSL certificates on Mac OSX. About time! I am tired of complaints from my mail server!
With the new Safari in Tiger, it is easy to import self-signed SSL certificates you may come across. Just click Show More in the alert box, and then drag the certificate icon to a folder or your desktop. Then double-click the certificate, and Keychain Access will prompt to import it. Select the "X509Anchors" keychain from the Select box and click OK. Finally, enter your admin password to allow that keychain to be modified.
For example, my university has a self-signed SSL certificate for their IMAP server, and Mail.app constantly complains about this. So I pointed Safari at the IMAP SSL port eg: https://imap.ufl.edu:993/ to grab and import the certificate into the system keychain. Now when I start up Mail.app, it doesn't complain when connecting securely to my school's IMAP server
05.12.2005 08:13
OpenInventor LineSet
This is a quick exploration of the OpenInventor/Coin LineSet (man SoLineSet).
I want to be able to simplify my usage of IndexedLineSet when I am not
reusing vertices and I also want to be able to do reasonable effecient
scatter plots so I will take a look at PointSet.
I do not know if PointSet is in SGI's OpenInventor, but I am hoping
that it is.
For LineSet, here is the example straight out of the Coin documentation:
For LineSet, here is the example straight out of the Coin documentation:
#Inventor V2.1 asciiThe above example makes three disconnected lines. What if we have the simpler case of just one line for all the vertices that we specify? All you have to do is change the numVertices to be -1!
Separator { Coordinate3 { point [ 0 0 0, 1 1 1, 2 1 1, 2 2 1, 2 2 2, 2 2 3, 2 3 2, 2 3 3, 3 3 3 ] } LineSet { numVertices [ 3, 4, 2 ] } }
#Inventor V2.1 asciiNow for a PointSet (man SoPointSet):
Separator { Coordinate3 { point [ 0 0 0, 1 1 1, 2 1 1, 2 2 1, 2 2 2, 2 2 3, 2 3 2, 2 3 3, 3 3 3 ] } LineSet { numVertices [ -1 ] } }
#Inventor V2.1 asciiWhy is it that we had a separate DotCloud application back in 1996-1997? Did Lew Hitchner all sorts of features that I never knew about (which is probably true) or was it a more fundamental problem with OpenInventor not having a PointSet back then?
Separator { Material { diffuseColor [ 1 0 0, 0 1 0, 0 0 1, 1 1 0, 1 0 1, 1 1 1, 1 0.8 0.6, 0.6 0.8 1 ] } MaterialBinding { value PER_PART }
Coordinate3 { point [ -1 1 0, -1 -1 0, 1 -1 0, 1 1 0, 0 2 -1, -2 0 -1, 0 -2 -1, 2 0 -1 ] }
DrawStyle { pointSize 3 }
PointSet { } }
05.12.2005 06:55
Another title about 'stuff'
pyparsing
got a positive but terse review. More
python parsing tools
POP and IMAP with python
Do not forget to download the latest firefox. 1.0.4 is out.
Dutch universities are going open. However, the actual website is all in dutch: DAREnet.
Kraftwerk
POP and IMAP with python
Do not forget to download the latest firefox. 1.0.4 is out.
Dutch universities are going open. However, the actual website is all in dutch: DAREnet.
Kraftwerk
05.11.2005 18:49
Wireless woe
So I can't seem to get a reliable wireless connection to the
basestation about 15 feet away. I think there are just too many base
stations all trying to use the same freqeuencies. Is there a decent
tool for Mac OSX that will tell me the real scoup on what it sees?
I am looking right now at how to do a funky self join in SQL. We will see if I can figure it out.... after about 15 minutes, here is what I have. I am guessing that there probably is a much better (read more efficient) way to rewrite this. First the table schema:
I am looking right now at how to do a funky self join in SQL. We will see if I can figure it out.... after about 15 minutes, here is what I have. I am guessing that there probably is a much better (read more efficient) way to rewrite this. First the table schema:
.schema mag CREATE TABLE mag ( samplename VARCHAR(40), treatment REAL, csd REAL, intensity REAL, dec REAL, inc REAL, timeMeasured TIMESTAMP, type VARCHAR(10), operator VARCHAR(20), depth REAL, cruise VARCHAR(20), corenum INTEGER, coretype VARCHAR(1), corehalf VARCHAR(1) , section INTEGER, sectionoffset REAL);I want arm from the table and irm from the table for the same samplename. But I want the arm to always be from the same treatment of 100. This makes a table alias ARM and a table alias IRM table which I bring together:
SELECT arm.samplename,arm.intensity,irm.intensity FROM mag arm INNER JOIN mag irm ON arm.samplename=irm.samplename WHERE arm.type='arm' AND irm.type='irm' AND arm.treatment=100 AND arm.corenum=5 ORDER BY arm.depth;Which gives results like this with the middle column being ARM 100 and the right column being IRM.
... bp04-5gw-s2-126|0.0000317600|0.0014030000 bp04-5gw-s2-129|0.0000277200|0.0011360000 bp04-5gw-s2-132|0.0000372000|0.0014940000 bp04-5gw-s2-135|0.0000351100|0.0013400000 bp04-5gw-s2-138|0.0000301600|0.0011740000 bp04-5gw-s2-141|0.0000346500|0.0014260000 bp04-5gw-s2-144|0.0000361300|0.0014160000
05.11.2005 16:29
mmap in python
Terry Fong and Marsokhod in the news: Overview of NASA's robots at BoingBoing. Thanks to ACD.
Lots of great data at the California Geological Survey: southern region quads. Thanks Roi! Check out San Diego on the right side: pdf
Wow. I did not know that python had easy access to mmap. This bodes well for my ideas on writing a generic low level driver for reading segy files quickly (maybe with writing too). I had been thinking that I needed some sort of low level C/C++ code to allow fast access into a file, but not if I can mmap in the file!
It is funny that this article is trying to get around a lack of solid IPC in MS Windows. I did not realize that windows had mmap. Now I need 64 bit mmap.
Communicate between processes using mmap
More on matplotlib which I think is more flexible than Gnuplot.
Lots of great data at the California Geological Survey: southern region quads. Thanks Roi! Check out San Diego on the right side: pdf
Wow. I did not know that python had easy access to mmap. This bodes well for my ideas on writing a generic low level driver for reading segy files quickly (maybe with writing too). I had been thinking that I needed some sort of low level C/C++ code to allow fast access into a file, but not if I can mmap in the file!
It is funny that this article is trying to get around a lack of solid IPC in MS Windows. I did not realize that windows had mmap. Now I need 64 bit mmap.
Communicate between processes using mmap
More on matplotlib which I think is more flexible than Gnuplot.
05.11.2005 06:25
Python/emacs
An emacs
mode for spyce. So I really don't understand the lisp code, but
it sounds interesting and it really is a short snippet. This follows
after I was thinking yesterday about emacs modes.
Hidding mp3's in gifs
Python challenge solutions, part 1
Flashkit - If you use flash, this is supposed to have lots of cool stuff. I would like to play with flash someday. It looks very cool. Need more time/day!
Hidding mp3's in gifs
Python challenge solutions, part 1
Flashkit - If you use flash, this is supposed to have lots of cool stuff. I would like to play with flash someday. It looks very cool. Need more time/day!
05.10.2005 19:36
A9.com Block View
Saw an interesting article in the June Discover: "Come Fly With Me."
It talks about Google's Keyhole flyover software that uses keyhole
satellite data and the A9.com (Amazon) Block View system where you can
go up and down a block.
Here is Frankie Johnnie & Luigi Too in Mountain View where Roy, Ruwan and I have met up many times. I think there should be a wiki for posting funny pictures from Block View. Find anyone you know driving by in the picture?
More on bash by Ken Steube at the SDSC.
I really want a nicer OpenInventor emacs mode. Maybe someday I will read this mode tutorial and write a real mode. I did write a mode for Arc Macro Language (AML), but I have long since lost the code. Maybe it is still hiding on the USGS WR sierra machine.
Here is Frankie Johnnie & Luigi Too in Mountain View where Roy, Ruwan and I have met up many times. I think there should be a wiki for posting funny pictures from Block View. Find anyone you know driving by in the picture?
More on bash by Ken Steube at the SDSC.
I really want a nicer OpenInventor emacs mode. Maybe someday I will read this mode tutorial and write a real mode. I did write a mode for Arc Macro Language (AML), but I have long since lost the code. Maybe it is still hiding on the USGS WR sierra machine.
05.10.2005 15:40
Schneier on Cryptography
Bruce Schneier
on Cryptography at Security Focus.
YellowIcon - another free icon site.
Yellowstone Rated High for Eruption Threat
Free Wendy's frosty this weekend
Tomsnetworking on how to crack wep - Remember, just because you are running wep someone may be listening to your traffic.
YellowIcon - another free icon site.
Yellowstone Rated High for Eruption Threat
Free Wendy's frosty this weekend
Tomsnetworking on how to crack wep - Remember, just because you are running wep someone may be listening to your traffic.
05.10.2005 09:15
Python data visualization
The linux gazette has an interesting article: Python for
scientific use. Part I: Data Visualization
WordPress 1.5.1 released. This to try after I graduate.
Scary: MIT Profs, Colleagues Propose Plan For Nuclear Energy. What is the true solution for long term global energy needs?
Previously Unknown Fault Provides New Insights On Himalayan Mountain Building
Surf Not Up For Palaeozoic Creatures - New Model Reveals Ancient Sea Was A Giant Lake
Google maps weblogs
10 Big Myths about copyright explained. I should read this.
More bash craziness. Part 4. If I was still doing my whole database work from bash, I would devour this article. As it is, I'll stick to python. But Ben writes very well and bash is my favorit shell (I never want to go back to csh,tcsh,or the most evil or evil's - DOS)
More on knoppix... what to use when you can't deal with MS windows any more.
Work on Opportunity to get it out of a dune
WordPress 1.5.1 released. This to try after I graduate.
Scary: MIT Profs, Colleagues Propose Plan For Nuclear Energy. What is the true solution for long term global energy needs?
Previously Unknown Fault Provides New Insights On Himalayan Mountain Building
Surf Not Up For Palaeozoic Creatures - New Model Reveals Ancient Sea Was A Giant Lake
Google maps weblogs
10 Big Myths about copyright explained. I should read this.
More bash craziness. Part 4. If I was still doing my whole database work from bash, I would devour this article. As it is, I'll stick to python. But Ben writes very well and bash is my favorit shell (I never want to go back to csh,tcsh,or the most evil or evil's - DOS)
More on knoppix... what to use when you can't deal with MS windows any more.
Work on Opportunity to get it out of a dune
05.09.2005 14:20
Joann Stock on the Gulf of California
Got to hear Joann Stock talk about the Gulf of California today. Great talk!
unrealid
Spacial Metaphore - debates about finder and all
VideoEditing
Working today on drying out my samples to get weight percent water. Not going as fast as I hoped. I now have two vacuum balls full of samples. I put some fresh dryright in one of them to see if that makes a difference.
Photos of drying.
unrealid
Spacial Metaphore - debates about finder and all
VideoEditing
Working today on drying out my samples to get weight percent water. Not going as fast as I hoped. I now have two vacuum balls full of samples. I put some fresh dryright in one of them to see if that makes a difference.
Photos of drying.
05.09.2005 05:22
ALIA-KM0506 Samoa Expedition
Check out the web page for the Samoa expedition:
ALIA
Someone on the fink dev mailing list posted a picture that was being served up by gopher. Yikes. Flashbacks to 1993.
gopher://sdf.lonestar.org/00/users/newmanbe/Random_Pictures_I_Have_Made/fin=kmap.png
ALIA
Someone on the fink dev mailing list posted a picture that was being served up by gopher. Yikes. Flashbacks to 1993.
gopher://sdf.lonestar.org/00/users/newmanbe/Random_Pictures_I_Have_Made/fin=kmap.png
05.08.2005 21:00
Gantt charts and other notes
GanttProject
Using Ant rather than GNU Make. Will I ever give up make files?
Did some more IRM's today on the Santa Barbara cores. Finished up core 6 and 1. Started those two sets of cubes drying in the fume hood. Need to get them setup in vacuum drier to make sure.
Two notes from Becca about xraying... news from the Gulf of Mexico
Using Ant rather than GNU Make. Will I ever give up make files?
Did some more IRM's today on the Santa Barbara cores. Finished up core 6 and 1. Started those two sets of cubes drying in the fume hood. Need to get them setup in vacuum drier to make sure.
Two notes from Becca about xraying... news from the Gulf of Mexico
Just read your blog and we got some software from the xray guy on the cruise - Barry. He uses some free software called ImageJ
http://rsb.info.nih.gov/ij/download.html
it is medical software but is very useful for viewing xrays in tif format and Barry wrote a macro using the pixel intensity (~density) to calculate out the cracks and voids in the core to get the true length. I have not really used it yet but he swears by it so definetly worth a look!
Barry brought a ct and a really really high res xray machine that he built on the boat - it can xray cores that are stil under pressure in an autoclave! and then if that is not enough he can also do "time lapse" measurements of the core degassing as the pressure is released slowly. All sorts of techno gradgetry is going on!
05.07.2005 13:40
4D visualization/capture
The
Hungry Mind: Prof. Avideh Zakhor on being a science nerd in Iran, why
Larry Summers made her mad, and what the heck a 4D model is.
Interesting content is at the bottom. She is trying to capture both
the spacial component along with the temporal changes. Here are the
key links. Some of her work is here:
UC Berkeley Video and Image Processing Lab
Next Generation, 4-D Distributed Modeling and Visualization - vismuri
An older Slashdot article: Automatic 3D Reconstruction of Scenes about the instant scene modeler which looks like they are using like the gray point system, no?
I need to look through the available open source 3D software again to see what the possibilities are as I do not think the Ames Stereo Pipeline will be released any time soon.
Quicktime Pro 6 Key with most of 7 - Use a backup of Quicktime Pro 6 on a machine with 7. The old app will use the new codecs.
15 things you can do with RSS. Don't forget to scroll down to the comments too.
Learning morse code with a binary tree. It has been about 17 years since I took the morse code test. Back then I only passed because I guessed what the scentence was from the few letters that I got right. It would be fun to be really good at morse code. I would love to be able to practice by entering text into my computers in morse via a keyer. That would be a fun and very simple microcontroller project.
Back to processing the new core photo...
UC Berkeley Video and Image Processing Lab
Next Generation, 4-D Distributed Modeling and Visualization - vismuri
An older Slashdot article: Automatic 3D Reconstruction of Scenes about the instant scene modeler which looks like they are using like the gray point system, no?
I need to look through the available open source 3D software again to see what the possibilities are as I do not think the Ames Stereo Pipeline will be released any time soon.
Quicktime Pro 6 Key with most of 7 - Use a backup of Quicktime Pro 6 on a machine with 7. The old app will use the new codecs.
15 things you can do with RSS. Don't forget to scroll down to the comments too.
Learning morse code with a binary tree. It has been about 17 years since I took the morse code test. Back then I only passed because I guessed what the scentence was from the few letters that I got right. It would be fun to be really good at morse code. I would love to be able to practice by entering text into my computers in morse via a keyer. That would be a fun and very simple microcontroller project.
Back to processing the new core photo...
05.07.2005 11:13
Preventing owners from data
The same goes with cars and ships. I think it is not right to
disallow the owner of a car, ship, or plane from the diagnostic data
for the vehicle or certain parts.
Engine Monitor Data Change Upsets Owners
Cracking the realestate code
Engine Monitor Data Change Upsets Owners
Cracking the realestate code
05.07.2005 10:27
Reshooting BPSIO-04 gravity cores
Yesterday Liz helped me reshoot the BPSIO-04 cores. We first scraped
the surface of the cores to remove wire drag marks and the older
oxidized surface. The hope is that this would reveal many more
features in the core. I really should have done this when I split the
cores, so I will definitely remember that for next time I collect
cores!
To do the scraping, we removed the plastic marker plugs and used a piece of metal lightly dragged across the surface. We tried both with and without the fluorescent lights and think that without them produces a better image.
Photos of the process were taken with the paleomag digital camera - an Olympus Camedia Z3000 Zoom 3.3 MPixel model that is now a couple years old.
To do the scraping, we removed the plastic marker plugs and used a piece of metal lightly dragged across the surface. We tried both with and without the fluorescent lights and think that without them produces a better image.
Photos of the process were taken with the paleomag digital camera - an Olympus Camedia Z3000 Zoom 3.3 MPixel model that is now a couple years old.
05.06.2005 08:49
mosaicing xrays
I have been thinking about the issues with digital xrays of cores.
What we have done works pretty well, but we can always do better.
There are several goals that I have in mind for processing xrays.
This morning, I ran into a document on the web about compositing xray images from paintings using a system called VIPS:
Working With Digital X-ray Images: Some practical image manipulation using VIPS/ip 7." by Joseph Padfield. Some issues are the same, some are not. It would be great to be able to do a good job with whole round or half round cores by correcting for both thickness and xray emmission issues(e.g. where on the plate the pixel comes from).
- Make nice mosaics for publications
- Make mosaics that best show off all of the features in the core
- Be able to make quantative statements about xray density
This morning, I ran into a document on the web about compositing xray images from paintings using a system called VIPS:
Working With Digital X-ray Images: Some practical image manipulation using VIPS/ip 7." by Joseph Padfield. Some issues are the same, some are not. It would be great to be able to do a good job with whole round or half round cores by correcting for both thickness and xray emmission issues(e.g. where on the plate the pixel comes from).
05.06.2005 07:51
Network toolkits and depth level in Open Inventor
Terry Fong based me a CMU tech report yesterday that is pretty cool.
Jay Gowdy compares network transport packages for robotics.
A Qualitative Comparison of Interprocess Communications Toolkits for Robotics
It is missing a some of the things that I consider with network toolkits and it is a bit dated at 2000, but definitely a great read.
On another note, I just added a draggable depth line to my xcore displays. I was getting annoyed with trying to figure out what depth an object is at and I did not want to go write a ray pick call back. This is based on the floating ocean for my Eel River / Humboldt 2002 project.
A Qualitative Comparison of Interprocess Communications Toolkits for Robotics
It is missing a some of the things that I consider with network toolkits and it is a bit dated at 2000, but definitely a great read.
On another note, I just added a draggable depth line to my xcore displays. I was getting annoyed with trying to figure out what depth an object is at and I did not want to go write a ray pick call back. This is based on the floating ocean for my Eel River / Humboldt 2002 project.
#Inventor V2.1 ascii # Small IV file to give a depth reference line so that I stop # spending so much time guessing depths. Separator { Separator { Transform { rotation 0 -1 0 1.57 } DEF +2 Translate1Dragger {} } Separator { Translation { translation -20 0 5 } Scale { scaleFactor .3 .3 .3 } RotationXYZ { axis X angle 1.57 } Text3 { string "Depth in cm" = Calculator { C 0 0 0 = USE +2 . translation expression "oa = -C[0]" } . oa } } Translation { translation 5 -.2 0 } Transform { translation 0 0 0 = Calculator { C 0 0 0 = USE +2 . translation expression "oA = vec3f(0,0,C[0])" } . oA
} Coordinate3 { point [ 0 0 0, 300 0 0, ] } # Coord3 IndexedLineSet { coordIndex [ 0, 1, -1, ]} # IndexedLineSet }
05.06.2005 05:16
Mars Polar Lander found?
An article from Sky and Telescope: Mars Polar
Lander may have been found
Flash timeline
Sparklines with python - those weird little inline graphics.
JWZ on Picturetile part 2.
tcpflow
An embedded view of the Mac mini, Part 3: Rapidly prototype an embedded application at IBM dev works
Flash timeline
Sparklines with python - those weird little inline graphics.
JWZ on Picturetile part 2.
tcpflow
An embedded view of the Mac mini, Part 3: Rapidly prototype an embedded application at IBM dev works
05.04.2005 13:47
python stuff
tclpython
- Calling python code from within Tcl.
Contact sheet of images using PIL. Maybe this will be better than what I was doing with JWZ's code that would not work do to a broken image magick on OSX.
Search Resumes for the Mars Polar Lander (MPL)
Contact sheet of images using PIL. Maybe this will be better than what I was doing with JWZ's code that would not work do to a broken image magick on OSX.
Search Resumes for the Mars Polar Lander (MPL)
05.03.2005 02:49
Dulles airport
It's 2:40 AM PST in the Dulles airport with 2 hours until my next
flight, so it is time to catch up a on few links sitting in my
laptop's dock.
Lake Tahoe makes the top of NSF's news.
Z800 3D Visor - Yet another head mounted display. No head tracking?
Freephotoshop.com - Brushes, etc.
Geologists' Find Is Evolutionary
Hackers guide to QT
How well do you know python - Say what? Maybe it is that I am reading this in the middle of the night.
Lake Tahoe makes the top of NSF's news.
Z800 3D Visor - Yet another head mounted display. No head tracking?
Freephotoshop.com - Brushes, etc.
Geologists' Find Is Evolutionary
Geologists believe that Earth experienced at least three global ice ages from 800 to 580 million years ago (mya) during the Neoproterozoic Era, at the end of the Precambrian Age. In most places, evidence of the glaciations has disappeared, but traces remain in several areas with "cap carbonates," thin layers of limestone containing distinctive ratios of carbon isotopes. "When the ice melts, the oceans rapidly precipitate carbonate, which appears in the sedimentary rock," explains one of the geologists, Daniel Condon, a postdoctoral fellow in geology/geochemistry. "Cap carbonates are very unusual and restricted to glacial periods in the Neoproterozoic."GCC 4.0 review
Hackers guide to QT
How well do you know python - Say what? Maybe it is that I am reading this in the middle of the night.
05.01.2005 06:59
why updatedb has been filling my disk
I have been dealing with daily find/sort issues stemming from the fink
locate/updatedb system. Here are some posts about this.
Hula Girl hot sauces
>>>> When I run updatedb from findutils in fink, I get this error: > >>> Does /usr/bin/find have the same problem? > >> It doesn't say it does. > > Sorry to bother everyone, but found a fix. I added fsdec to the list > of fstypes to prune, and it worked fine (/dev/fd/3 couldn't be stat'd) > Maybe we can still track down why it says the hard link count is > incorrect though; if it doesn't have that error there is a very nice > speed boost because it doesn't stat each file then. > Hey folks,
I'd like to add that this is a *critical* bug, and it occurs on 10.3, too -- maybe the new findutils version introduce the problem?
Today and yesterday my HD run full. The problem is that apparently, for updatedb /dev/fd/3 is an alias for /. So when updatedb scans my FS, it gets into an infinite recursion loop. In my case it created about 15 GB of temp files (indexing the "infinitely many files" on my HD), before it run out of HD space.
Running out of HD space is rather dangerous, if you notice it too late it can cause data loss (10.3 really tries to warn you about this, but if your machine is running while you are absent, as in my case, you can be too late)...
Hula Girl hot sauces