03.31.2005 15:05
Processing the core photos
After copying the core xray images onto my computer, it is time to
dive into processing them. This blog entry to may contain some twists
and turns as I figure out how to do this. I will be using Photoshop
CS for Mac. I will use bp04-006-011.tif as my example. This is the
second frame down the first set. Here are my notes on this frame.
For each one, look at the image and see if the L and R letters are right side of and L on the left/R on the right. Use Image->Rotate Canvas->180 to flip to right side up and Image->Rotate Canvas->Flip Canvas Horizontal to fix a left right swap if they exist. Or you can just do Image->Rotate Canvas->Flip Vertical if they all have both of these transforms. This is the reason that you put the LR lead letters on the core (which I forgot to do and Liz remembered after a couple.) Having a clear
FIX: image 03- dir
Now use the Text Tool (T) to add descriptions to the tif file for each of the core sections and label the depth for each of the markers. Make sure to use full white as your color or you labels will disappear later on in the process. I used 30pt Aerial Rounded MT Bold. Save the file as a Photoshop PSD to allow saving the text as a layer. Note that placement is not critical since a follow on step for each core section is to mask out the other sections in the image.
FIX: image 04-dir
This on hold until I get brain power back from the flu.
Aproximately 35-75 cm 5GAS2 on the right, 2GAS1 center, 1GAS2 on the left The top of 2GAS1 is in this image.FIX: image 02
For each one, look at the image and see if the L and R letters are right side of and L on the left/R on the right. Use Image->Rotate Canvas->180 to flip to right side up and Image->Rotate Canvas->Flip Canvas Horizontal to fix a left right swap if they exist. Or you can just do Image->Rotate Canvas->Flip Vertical if they all have both of these transforms. This is the reason that you put the LR lead letters on the core (which I forgot to do and Liz remembered after a couple.) Having a clear
FIX: image 03- dir
Now use the Text Tool (T) to add descriptions to the tif file for each of the core sections and label the depth for each of the markers. Make sure to use full white as your color or you labels will disappear later on in the process. I used 30pt Aerial Rounded MT Bold. Save the file as a Photoshop PSD to allow saving the text as a layer. Note that placement is not critical since a follow on step for each core section is to mask out the other sections in the image.
FIX: image 04-dir
This on hold until I get brain power back from the flu.
03.31.2005 14:01
Undersea Landslides in Monterey Canyon
In the May 2005 Discover, page 88 is a one pager on underwanter
landslides in the Monterey Canyon, CA.
The butterflies are still going north through San Diego. Today I identified one as a Monarch as it got close. It looks like it is now a different butterfly species coming through by the thousands. UPDATE: The news now says they are the painted lady butterfly.
I heard today that there is a group of people in the San Joaquin valley that has an extra stomach muscle. Wow. However, I can not seem to find any information on this via google. If you know more, email me!
The butterflies are still going north through San Diego. Today I identified one as a Monarch as it got close. It looks like it is now a different butterfly species coming through by the thousands. UPDATE: The news now says they are the painted lady butterfly.
I heard today that there is a group of people in the San Joaquin valley that has an extra stomach muscle. Wow. However, I can not seem to find any information on this via google. If you know more, email me!
03.31.2005 06:44
xraying bpsio cores
Yesterday, Liz and I xrayed the BPSIO-04 archive half cores. The
results seem excellent. Many thanks to Liz J. and the WindanSea Veterinary
Clinic for all the help!
First some ground rules about the place we did the xrays. These people are super nice. Pay them whatever they ask: It is absolutely worth it! Make sure to stay out of there way and if they need the xray machine for an animal, make sure they can use it without delay. Be very careful with the xray screens. They are very expensive. And cleanup after yourself!
After using their digital xray machine, I never want to get near the SIO dental xray machine in bottom floor of Hubbs Hall.
To give you a taste, here is a quick thumbnail of one xrays before it has been adjusted in any way. I tool Galerie to generate a quick preview of the raw images. I will release the full originals once I have this thing published. This is my first attempt at a quick water mark. The tip there is just make a transparent tif in Photoshop with black text. The white came through at 100%. Ouch. Note that the core is upside down and flipped left-right in the scan.
One note is that Pic2Icon on the Mac did not make icons for these images in the folder I have. After copying images from CD, you will need to "chmod 644 *.tif" the images before you can make icons.
With the raw 16-bit unprocessed tif which Galerie turned into a smaller 8-bit tiff, it is impossible to see anything in the core. But it is there! It just needs some processing. The only drawback to this xray system is that the digital scanner does not know the settings that were on the xray machine. Liz reminded me to write those down. Our first 5 frames were testing out the settings to see what works best.
Starting out from the beginning, here is how I did the xrays...
On 29-Mar-2005, I brought out each core from the Reefer (that's the marine scientists name for the refrigerator) to mark the cores for xraying with a trick Liz picked up from Neal. The idea is to put plastic pegs into the core. These pegs displace sediment so there is a square mark on the xradiograph where more xrays go through to the plate. I do not really know what Neal's strategy for using these things, but here is what I did.
I only have six cores so this might not be good for larger projects. At the top of each core, I placed a number of pegs that matches the core number. On the top of the cores in the above photo, you can see 5 pegs in the top of core 5 on the left and 1 each on cores 1GA-S1 and 1GA-S2 in the middle and right respectively. 1GA-S1 is the little short top part of core 1. Note that G is for "Gravity" and A is for "Archive". On the tops of these pegs I wrote the core and section numbers. Unfortunately, due to differing properties for the core tops, I was not able to put all of them at a 001 cm offset. Please do not count on the position of the top markers.
Then down core I placed a marker at each 10 cm offset. Remember that I do not use absolute depths when working with core data. I feel uncomfortable with doing that translation early in the work. I use a backed database lookup to make that translation from sample name (which includes the offset cm) to a distance from the overall cores top (which is often not the bottom/water interface).
Each xray image can cover about 38 cm. To help decrypting which marker is for what offset, I use a strategy of adding a right hand marker at 30 cm intervals (e.g. section offsets of 030, 060, 090, 120cm). This also helps make sure I don't have the cores flipped left to right.
The next task was getting the xray machine setup. Here is a quick intro to using the xray machine. You will need to setup the current used (milli Amps - mA), the exposure time (in seconds), and the xray energy level (in kilo Volts - kV). This will change based on the material type and the thickness. Liz used different settings for the png cores since they are in 4 inch inner diameter (4" ID) PVC liner, while my cores are in 3.25" ID. Liz used 70kV, while mine were mostly 60kV.
Note the exposure times here. When using the 40kV machine in the basement of Hubbs Hall I got very poor results. My notes do not seem to say how many Amps that that little dental tube based xray put out, but the exposures were 3 and 4 minutes
There were already 5 PNG pictures in the "Scripps" animal listing of the xray program, so I started numbering from 006. But it turns out that they get renumbered when you export to tif files. Save# is the file name index on the cd, while Scripps# is what you see on the computer screen inside the xray scanning program. Here are my exposure settings.
Next, lay out the cores on the table. We were able to put 3 cores side by side on the table. Use the left red button on the overhead to turn on a light that shows you where the xrays will be hitting. We used the maximum window. You can see that there was still room for another core in the lighted area.
Slide the plate under carefully as it can break. The vet depends on this for taking care of animals and they are reasonably expensive. Once the place is under the core and the light hits the whole thing, you are ready to take an xray. Exit the room and close the lead plated sliding door. Use the foot trigger to trip the xray machine. You can then remove the plate and wipe it dry of any liquid from the cores.
Now you are ready to scan the xray. Here is the scanner with a screen loaded, but the door open.
Take the plate/screen over to the xray scanner in the far back corner. Place the screen with the notch securely locked into the guide at the top right of the screen. Get help if you do not know how to properly insert and remove screens into the magnetic holder. Then close the door.
Go to the PC and select the "Scripps" animal. Select scan. Pick the largest size and it will read the xray. Hit cancel if something is not right. As the image scans, it will not look very good. Don't panic. As long as you see your core there, things may be okay. Once it has homed the slide and finished, close the scanning pannel. Then double click on the latest image. You can drag the middle mouse button left/right to adjust the contrast to see if there really is structure in the image of your core. The machine scans 16-bit gray scale images, so quite a bit of detail may be hidden in there. Here is the screen after adjusting the contrast.
After you go through all of your cores, select all of your new xray images and do a File->export. Save them to a folder on the desktop. Then go to the Start menu -> Programs -> Roxio. Create a data cd project. Name the part of the project after the date as you see fit. I like to leave the date part intact. Then insert a CD-R into the top CD drive and burn your project.
At this point you have your xrays and after you clean up the xray room are good to go. Make sure that you are all caught up with billing before leaving.
Don't forget to check out the amazing fish tank in the lobby!
In a later entry I will document how to convert these xray frames into something usable for research. I now have 17 images that need processing. Here is the ImageMagick identify description of one image:
First some ground rules about the place we did the xrays. These people are super nice. Pay them whatever they ask: It is absolutely worth it! Make sure to stay out of there way and if they need the xray machine for an animal, make sure they can use it without delay. Be very careful with the xray screens. They are very expensive. And cleanup after yourself!
After using their digital xray machine, I never want to get near the SIO dental xray machine in bottom floor of Hubbs Hall.
To give you a taste, here is a quick thumbnail of one xrays before it has been adjusted in any way. I tool Galerie to generate a quick preview of the raw images. I will release the full originals once I have this thing published. This is my first attempt at a quick water mark. The tip there is just make a transparent tif in Photoshop with black text. The white came through at 100%. Ouch. Note that the core is upside down and flipped left-right in the scan.
One note is that Pic2Icon on the Mac did not make icons for these images in the folder I have. After copying images from CD, you will need to "chmod 644 *.tif" the images before you can make icons.
With the raw 16-bit unprocessed tif which Galerie turned into a smaller 8-bit tiff, it is impossible to see anything in the core. But it is there! It just needs some processing. The only drawback to this xray system is that the digital scanner does not know the settings that were on the xray machine. Liz reminded me to write those down. Our first 5 frames were testing out the settings to see what works best.
Starting out from the beginning, here is how I did the xrays...
On 29-Mar-2005, I brought out each core from the Reefer (that's the marine scientists name for the refrigerator) to mark the cores for xraying with a trick Liz picked up from Neal. The idea is to put plastic pegs into the core. These pegs displace sediment so there is a square mark on the xradiograph where more xrays go through to the plate. I do not really know what Neal's strategy for using these things, but here is what I did.
I only have six cores so this might not be good for larger projects. At the top of each core, I placed a number of pegs that matches the core number. On the top of the cores in the above photo, you can see 5 pegs in the top of core 5 on the left and 1 each on cores 1GA-S1 and 1GA-S2 in the middle and right respectively. 1GA-S1 is the little short top part of core 1. Note that G is for "Gravity" and A is for "Archive". On the tops of these pegs I wrote the core and section numbers. Unfortunately, due to differing properties for the core tops, I was not able to put all of them at a 001 cm offset. Please do not count on the position of the top markers.
Then down core I placed a marker at each 10 cm offset. Remember that I do not use absolute depths when working with core data. I feel uncomfortable with doing that translation early in the work. I use a backed database lookup to make that translation from sample name (which includes the offset cm) to a distance from the overall cores top (which is often not the bottom/water interface).
Each xray image can cover about 38 cm. To help decrypting which marker is for what offset, I use a strategy of adding a right hand marker at 30 cm intervals (e.g. section offsets of 030, 060, 090, 120cm). This also helps make sure I don't have the cores flipped left to right.
The next task was getting the xray machine setup. Here is a quick intro to using the xray machine. You will need to setup the current used (milli Amps - mA), the exposure time (in seconds), and the xray energy level (in kilo Volts - kV). This will change based on the material type and the thickness. Liz used different settings for the png cores since they are in 4 inch inner diameter (4" ID) PVC liner, while my cores are in 3.25" ID. Liz used 70kV, while mine were mostly 60kV.
Note the exposure times here. When using the 40kV machine in the basement of Hubbs Hall I got very poor results. My notes do not seem to say how many Amps that that little dental tube based xray put out, but the exposures were 3 and 4 minutes
There were already 5 PNG pictures in the "Scripps" animal listing of the xray program, so I started numbering from 006. But it turns out that they get renumbered when you export to tif files. Save# is the file name index on the cd, while Scripps# is what you see on the computer screen inside the xray scanning program. Here are my exposure settings.
Save# Scripps# current(mA) exposure time(s) energy(kV) 001 006 300 1/10 60 002 007 300 1/10 70 003 008 300 1/10 70 004 009 300 1/20 70 005 010 300 1/10 60 006 011 300 1/10 60 ... " " " " 017 022 300 1/10 60I decided that 300mA, 1/10th sec, and 60kV gave pretty good results. Setup the uni-master 325 to these setting after you have turned it on. Here is the console:
Next, lay out the cores on the table. We were able to put 3 cores side by side on the table. Use the left red button on the overhead to turn on a light that shows you where the xrays will be hitting. We used the maximum window. You can see that there was still room for another core in the lighted area.
Slide the plate under carefully as it can break. The vet depends on this for taking care of animals and they are reasonably expensive. Once the place is under the core and the light hits the whole thing, you are ready to take an xray. Exit the room and close the lead plated sliding door. Use the foot trigger to trip the xray machine. You can then remove the plate and wipe it dry of any liquid from the cores.
Now you are ready to scan the xray. Here is the scanner with a screen loaded, but the door open.
Take the plate/screen over to the xray scanner in the far back corner. Place the screen with the notch securely locked into the guide at the top right of the screen. Get help if you do not know how to properly insert and remove screens into the magnetic holder. Then close the door.
Go to the PC and select the "Scripps" animal. Select scan. Pick the largest size and it will read the xray. Hit cancel if something is not right. As the image scans, it will not look very good. Don't panic. As long as you see your core there, things may be okay. Once it has homed the slide and finished, close the scanning pannel. Then double click on the latest image. You can drag the middle mouse button left/right to adjust the contrast to see if there really is structure in the image of your core. The machine scans 16-bit gray scale images, so quite a bit of detail may be hidden in there. Here is the screen after adjusting the contrast.
After you go through all of your cores, select all of your new xray images and do a File->export. Save them to a folder on the desktop. Then go to the Start menu -> Programs -> Roxio. Create a data cd project. Name the part of the project after the date as you see fit. I like to leave the date part intact. Then insert a CD-R into the top CD drive and burn your project.
At this point you have your xrays and after you clean up the xray room are good to go. Make sure that you are all caught up with billing before leaving.
Don't forget to check out the amazing fish tank in the lobby!
In a later entry I will document how to convert these xray frames into something usable for research. I now have 17 images that need processing. Here is the ImageMagick identify description of one image:
identify bp04-001.tif bp04-001.tif TIFF 1748x2116 PseudoClass 65536c 7.1mb 0.490u 0:01To be continued...
03.31.2005 06:09
Morning links
Mars
airplane needs a magnetometer. CL Johnson and JS Gee would make
excellent Co-I's plus they have SIO E&M groups behind them with an
amazing lineup of smarts... Cande, Constables, Parker, Tauxe, YJ Yu,
and more. SIO is very close to MSSS and just a "short" drive to JPL.
Survey Finds Silver Contamination In North Pacific Waters - Silver could be a nice new tracer for ocean processes.
New Technique Uses Seismic 'Garbage' To View Earth's Interior
TES Dust page. No dust? Or just no data?
Python 2.4.1 released
Multimethods in Python. There is where you have two function foo(int) and foo(float) and they are called based on the type you pass to foo. Isn't this called overloading in C++ or is python crowding out my C++ knowledge?
Missed out on meeting J. Kennett yesterday. I was a little too tired to try to go meet him while he sampled over at ODP yesterday. Hopefully I can find him today.
Survey Finds Silver Contamination In North Pacific Waters - Silver could be a nice new tracer for ocean processes.
New Technique Uses Seismic 'Garbage' To View Earth's Interior
TES Dust page. No dust? Or just no data?
Python 2.4.1 released
Multimethods in Python. There is where you have two function foo(int) and foo(float) and they are called based on the type you pass to foo. Isn't this called overloading in C++ or is python crowding out my C++ knowledge?
Missed out on meeting J. Kennett yesterday. I was a little too tired to try to go meet him while he sampled over at ODP yesterday. Hopefully I can find him today.
03.29.2005 07:19
The Planets
Yesterday, I went and checked out the first day of Earth001 - The
Planets. 170 students! Wow that is big. The class gets bigger every
year and enrollment is maxed out. Too bad UCSD did not have the
interactive teaching technology working. A note to students:
I have been trying to build kdebase3-ssl. I had to kill it for battery life yesterday after about 3 hours of compiling. I started again in an xterm instead of FinkCommander.
A very silly shirt for your iPod mini.
MacJams.com - Creative Commons music. Have not had a chance to listen to any of this stuff. Any good ones you have found?
What is this new open search business? A9 Launches "Open Search" - Vertical Search, Syndicated
Put away your cell phones and PSP's. I know you have a cool new toy, but it is disrespectful to the students around you that are there to learn. And don't touch those backpacks until class is really over.Graham Kent was on NBC 7 this morning flying around on the VizCenter screen showing yesterday's Sumatra earthquake in Fledermaus.
I have been trying to build kdebase3-ssl. I had to kill it for battery life yesterday after about 3 hours of compiling. I started again in an xterm instead of FinkCommander.
A very silly shirt for your iPod mini.
MacJams.com - Creative Commons music. Have not had a chance to listen to any of this stuff. Any good ones you have found?
What is this new open search business? A9 Launches "Open Search" - Vertical Search, Syndicated
03.28.2005 15:31
Battling the HP Printer
Today Christie tried to help me battle the HP Printer in the lab. But
no luck. Everything is ghosty. It may be because my figure goes over
the size of the paper somehow. The folks printing before me had to
shrink their powerpoint to smaller than the paper size by two inches.
But I have not had this problem before with my previous posters. But
maybe it is because this is the first one where I altered the paper
size to 36x36. I used the custom size in the print advanced to get
that size. Now I don't seem to be able to undo the ghosting. Arg.
03.28.2005 10:32
Obligatory cat photo
I can't help but post this picture. Bailey just had a haircut and has
been acting like we live in Antarctica.
I am working on a poster of a zillion little plots of data from the BPSIO core. I am starting to drown in variations of plotting routines in python. I have plotFromDB, plotAgainstDepthFromDB, plot_declination, plot_2_nrm_int. First, I need to pick a naming convention and stick with it! I do not like _'s that much. But sometimes capitalization gets all weird. IDONNO! Secondly, plot_declination is not smart enough to handle AMS declinations. Whoops! Oh wait, it is able to handle AMS declinations. Just my API is not that stable so I don't remember where things are. Consider everything in manyplots_poster.py just a prototype with me learning how to cope with SQL tables and py gnuplot.
It is taking an hour to get all these plots placed correctly in the plot. I really need to automate the poster generation (which was my original plan). Too bad JWZ's Tetris based image placement perl program does not work on the Mac due to a bug inside of ImageMagick.
Ouch. Wireless in the UCSD library is best described as spastic.
I am working on a poster of a zillion little plots of data from the BPSIO core. I am starting to drown in variations of plotting routines in python. I have plotFromDB, plotAgainstDepthFromDB, plot_declination, plot_2_nrm_int. First, I need to pick a naming convention and stick with it! I do not like _'s that much. But sometimes capitalization gets all weird. IDONNO! Secondly, plot_declination is not smart enough to handle AMS declinations. Whoops! Oh wait, it is able to handle AMS declinations. Just my API is not that stable so I don't remember where things are. Consider everything in manyplots_poster.py just a prototype with me learning how to cope with SQL tables and py gnuplot.
It is taking an hour to get all these plots placed correctly in the plot. I really need to automate the poster generation (which was my original plan). Too bad JWZ's Tetris based image placement perl program does not work on the Mac due to a bug inside of ImageMagick.
Ouch. Wireless in the UCSD library is best described as spastic.
03.28.2005 07:07
TSA lied
How the TSA lied about using personal data
TSA == Transportation Security Administration
Boulder Streams in the Falkland Islands.
TSA == Transportation Security Administration
Boulder Streams in the Falkland Islands.
03.27.2005 13:02
ivcon
A while back, I worked on ivcon trying to get it to convert
OpenInventor or WRL (VRML) files to DXF so that they were much easier
to import into Blender, Lightwave, and Maya. Well, no luck since it
seemed to miss out on parts of the model. But before what I did is
lost, here it is. The sourceforge project for ivcon has changes made
post what the main author had done, but it is all MS Windows based.
They fixed it so that it is not allocating large buffers on the stack
which causes troubles on PPC/OSX, but works ok on Linux x86/MSWin. I
needed a Make system to build this and I wanted the MS Visual C++
cruft out of there. Here is the make file and I put up a tar ball
snapshot should anyone want to play with it. This Makefile is public
domain or GPL'ed. Your choice.
The original author's ivcon The source forge ivcon My Makefile and source tar ball: ivcon-sf1.0-kds-hack.tar.bz2The Makefile:
SRCS+=byu.cpp SRCS+=dxf.cpp SRCS+=gmod.cpp SRCS+=hrc.cpp SRCS+=iv.cpp SRCS+=ivconv.cpp SRCS+=main.cpp SRCS+=obj.cpp SRCS+=pov.cpp SRCS+=smf.cpp SRCS+=stl.cpp SRCS+=tds.cpp SRCS+=tec.cpp SRCS+=tmat.cpp SRCS+=tri.cpp SRCS+=txt.cpp SRCS+=ucd.cpp SRCS+=utils.cpp SRCS+=vla.cpp SRCS+=wrl.cpp SRCS+=xgl.cpp OBJS:=${SRCS:.cpp=.o} TARGETS:=ivconv mesh CXXFLAGS:= -O3 -funroll-loops -fexpensive-optimizations -DNDEBUG -ffast-math -mpowerpc targets: ${TARGETS} ivconv: ${OBJS} g++ $^ -o $@ ${CXXFLAGS} mesh: mesh.o g++ $^ -o $@ ${CXXFLAGS} clean: rm ${OBJS} ${TARGETS}
03.27.2005 12:45
TTN136B work
I wanted to have more cubes on hand that I will need measured just in
case I run into a now or never situation or some extra help. Sampling
photos: 2005Mar02-01TC-W-S1
03.27.2005 09:50
Weka 3.4.4
I just submitted a new fink info file into the cvs tree for
weka-3.4.4. I never figured out why 3.4.3 stopped building. Better
to just go for the new version.
03.26.2005 17:12
Updates to python code docs for BPSIO
Today I am working on pseudo-Thellier for core 5. I have updated the
pydoc on the web. Here are the ones that had the most change today or
are new.
afdemag_poster build_db pseudotWoohoo. Got new phones from Verizon today. It did take almost 2 hours for them to get us processed. Something about how our phones had "too many codes" which locked up 3 different sales peoples computers. We finally had to do all the paperwork by hand.
03.26.2005 10:56
OSX Hacking Tools
Mac OSX
hacking tools is a nice web page that explains many of the cool
little programs that are hiding in /usr/bin and /usr/sbin among the
1000 programs that are in there. Things like ditto are super handy.
Use ditto to copy things with their resource fork.
03.25.2005 22:01
Makefile for pmag
Just ran into Lisa's new pmag-1.8.4. I realized that I had not made a
nice make rule to build a tar without obj, executable, and tmp files.
So I wrote up something quickly this evening:
VERSION := ${shell cat VERSION} BASE := /tmp/pmag-${VERSION} tar: @echo @echo " *** Building pmage release: ${VERSION} ***" @echo rm -rf ${BASE}.tar* ${BASE} mkdir /tmp/pmag-${VERSION} (cd .. && cp -rp * ${BASE}) @echo @echo "Cleaning up the copy of the tree in ${BASE}" rm -f ${BASE}/bin/* find ${BASE} -name mypost\* | xargs rm -f find ${BASE} -name .DS_Store | xargs rm -f find ${BASE} -name fort\* | xargs rm -f find ${BASE} -name core | xargs rm -f find ${BASE} -name \*.bak | xargs rm -f (cd ${BASE}/src && make clean) @echo @echo "Building the package" (cd ${BASE}/.. && tar cf pmag-${VERSION}.tar pmag-${VERSION}) bzip2 -9 ${BASE}.tar @echo @echo "Done. Cleaning up." @echo rm -rf ${BASE} @echo mv ${BASE}.tar.bz2 . @echo @echo "Your tar is now available as pmag-${VERSION}.tar.bz2" @echo @echo "Remember to email Kurt to update the fink info file"
03.25.2005 20:34
fink packages and patches, MirPorts
MirPorts is another Mac
OSX packager that is just starting up. Which brought up licensing of
fink packages and patches. I was just wondering the other day how I
can make sure that my patches and packages are the as usable to
everyone as possible. Now I know from DRM's message.
Fink joined the metapkg project a few years ago, in which several open source packaging projects agreed to share porting tips and patches. In this spirit, you are welcome to use any of fink's patches in another packaging project.
03.25.2005 20:27
cwp-su troubles
It seems there are still problems with CWPROOT for seismic unix. I
really need a test suite that I can just run to prove that everything
works!
Whenever I try to feed suxmovie with some data, it fails withThanks to Charles for detailing the troubles.
sh: line 1: /sw/lib/cwp-su/bin/bin/xmovie: No such file or directory
I think that there's an extra "bin" in the path used to execute stuff. It appears as if CWPROOT is set incorrectly.
Stephen Hocking
Charles Williams willic3 at rpi.edu Date: Fri, 25 Mar 2005 Subject: [Fink-users] Caution about cctools-528.5
Hi Everyone,
I had some recent build problems that were due to the fact I had installed a newer version of the cctools package. I have since reverted back to the original version provided with Xcode and was able to build things after that. Since then, however, I have discovered some other problems with libraries that were built with cctools-528.5. I first noticed problems when linking with mpich2 libraries I had built myself, so I rebuilt those. This morning, I discovered a problem with the python2.4 library:
ld: /sw/lib/python2.4/config/libpython2.4.a(_sre.o) r_type field of relocation entry 1622 in section (__TEXT,__text) invalid ld: /sw/lib/python2.4/config/libpython2.4.a(_sre.o) stray relocation PPC_RELOC_PAIR entry (1623) in section (__TEXT,__text) ld: /sw/lib/python2.4/config/libpython2.4.a(_sre.o) r_type field of relocation entry 1624 in section (__TEXT,__text) invalid ld: /sw/lib/python2.4/config/libpython2.4.a(_sre.o) stray relocation PPC_RELOC_PAIR entry (1625) in section (__TEXT,__text) make[2]: *** [/Users/willic3/geoframe/build/bin/Darwin-7.x_ppc-shared,debug,mpi/ mpipython.exe] Error 1
It appears that libraries built with the newer version of cctools are not compatible with the version I have now. I'm afraid this means I'm going to be rebuilding a lot of fink-installed packages (I'm rebuilding python2.4 right now). The moral of the story is: Don't be impatient (as I was) and grab third-party developer tools that become the defaults. Fink's method of dealing with this is much better. The newer versions of the tools are only used when they are specifically requested. If you did install the newer cctools, you will probably run into some of the same problems I am having.
Charles
03.25.2005 12:15
appscript
Carol Stoker says
that she never said what was in that wild article last month about a
paper on Life on Mars supposedly coming out in Nature.
sqlobject sounds like a nice python package for using SQL databases. Can it work with with SQLite? Looks like the answer is yes.
Chain of Responsibility in python. Adding handlers is so much easier than in C++! Make sure to read the posted comments for some good improvements
Hey Becca, it looks like the Escape game you sent me is very popular with 800 hits all of a sudden! So far I have lasted 12 seconds with my track pad.
Grad Tools to help me finish my thesis?
There are lots of reviews of graphics cards lately and they are starting back into the multiple GPU world again, but what seems missing from reviews are any tests on the accuracy of rendering. I have noticed that many cards may be fast, but they are sacrificing quality of rendering for speed. That is one thing I liked about SGI IRIX based hardware - rendering was done right. Little artifacts are a big problem for scientific work. There needs to be an OpenGL conformance and quality section folks! Wildcat Realizm 800: 3Dlabs MultiGPU First Look
Mac unix tool of the day: fs_usage or see man fs_usage. Helpful to see what files are being messed with by a program. Whoops. Lost the link. Doh.
"Your success as a scientist depends upon your reputation - your reputation for caution, your reputation for correctness, for publishing the truth, for not making claims you can't substantiate. If your reputation gets tarnished then that affects your ability to be competitive in the scientific community."Appscript looks cool. How hard would this be to get into fink? This is a python-AppleScript bridge.
...
Berger, meanwhile, maintains that the story as originally reported was legitimate.
"Our story accurately portrayed what Dr. Stoker told a trusted group of insiders about her research and its implications," he said in the e-mail to MSNBC.com. "As it happened, we were on the leading edge of what turned out to be a month chock full of intriguing science findings about Mars."
sqlobject sounds like a nice python package for using SQL databases. Can it work with with SQLite? Looks like the answer is yes.
self._conn = sqlite.connect(self.filename)
Chain of Responsibility in python. Adding handlers is so much easier than in C++! Make sure to read the posted comments for some good improvements
Hey Becca, it looks like the Escape game you sent me is very popular with 800 hits all of a sudden! So far I have lasted 12 seconds with my track pad.
Grad Tools to help me finish my thesis?
There are lots of reviews of graphics cards lately and they are starting back into the multiple GPU world again, but what seems missing from reviews are any tests on the accuracy of rendering. I have noticed that many cards may be fast, but they are sacrificing quality of rendering for speed. That is one thing I liked about SGI IRIX based hardware - rendering was done right. Little artifacts are a big problem for scientific work. There needs to be an OpenGL conformance and quality section folks! Wildcat Realizm 800: 3Dlabs MultiGPU First Look
Mac unix tool of the day: fs_usage or see man fs_usage. Helpful to see what files are being messed with by a program. Whoops. Lost the link. Doh.
fs_usage - report system calls and page faults related to filesystem activity in real-time
03.24.2005 10:21
SQL help from Alex
Sometimes it is better to ask for help. In this case, that was
certainly true. Alex gave me a quick leg up with a task that was
troubling me. For my BPSIO-2004 paleomag cube data, I need to be able
to quickly list the maximum afdemag treatment applied to the samples
in a core. This then lets me do a quicker job of pseudo-Thellier.
SELECT max(samplename),max(treatment) FROM mag WHERE corenum=5 AND \ type='afdemag' GROUP BY depth ORDER BY depth;This query gives something like this:
... bp04-5gw-s2-126|80.00 bp04-5gw-s2-129|80.00 bp04-5gw-s2-132|80.00 bp04-5gw-s2-135|60.00 bp04-5gw-s2-138|100.00 bp04-5gw-s2-141|100.00 bp04-5gw-s2-144|80.00
03.24.2005 06:35
Misc things
Looks like the compiling trouble for cwp-su (seimic unix) are do to a
bad ld in a new cctools. More when I know more.
py2doc - Make graphviz dot files out of python code!
Like the X prise, NASA has a Centenial Challenge. Not that exciting (to me at least). How about a challenge to how we can stop wasting money on the space station and space shuttles.
elementplist reader for Mac OSX plist files.
Another martian dust devel> - This one going across the plains.
py2doc - Make graphviz dot files out of python code!
Like the X prise, NASA has a Centenial Challenge. Not that exciting (to me at least). How about a challenge to how we can stop wasting money on the space station and space shuttles.
elementplist reader for Mac OSX plist files.
Another martian dust devel> - This one going across the plains.
03.22.2005 17:34
ISIS continues
I don't want to spend too much more time going into what fink provides
and selling it, but it would be a huge win for the science community
to get ISIS into fink.
The idea behind why I use fink is that there are many macs and many programs to install them on. Especially when getting a machine ready for work in flightops. If I have to install 100 programs by hand on each machine that gets seriously painful. The flightops install that I use current installs about 800 packages from fink.
With fink, it is easy to check for new versions of packages, bug fixes, and new packages. Each package has a person who watches the that package for updates. They then update the info file with the new version number, md5sum, test it, and then put the info file into the fink system.
For users to stay up to date, they do not have to check each package. They simply to a "fink selfupdate". Then fink can say what version of a package is installed and what is the latest available. This way users do no have to learn a different method of install for each application that they use. They get a unified view of what is available and installed.
If you have used debian linux, fink uses debian at the underlying level.
If you have not tried fink, you might want to give it a try. Install fink and fink commander, then in fink commander, under preferences -> fink, turn on unstable to get all of the packages.
Then if you don't like it, just junk the /sw tree and fink will be off of your system.
We have >100 fink users here at SIO doing everything from geology, biology, database design, engineering, etc.
fink is not perfect, but it does fill a huge need. For one particular program it may not be obvious why fink is helpful, but for a whole system it makes a big difference.
The goal is not to impact the way you normal do things, but for the fink community to help support/distribute programs like ISIS.
03.22.2005 11:13
AF demag of an IRM
Jason gave me a quick into to IRM yesterday and I blasted 10 samples
from core 6. I then did an afdemag on one to see how close to
demagetizing the sample I could get. Here is the data:
bp04-6gw-s1-094.0 0.0 0.3 3.468E-6 139.6 35.3 3/17/05;9:04 PM;G34;1 afdemag schwehr ... bp04-6gw-s1-094.0 40.0 0.4 1.083E-6 125.4 21.7 3/17/05;9:14 PM;G34;1 afdemag schwehr bp04-6gw-s1-094.0 50.0 0.5 6.843E-7 110.4 6.0 3/17/05;9:15 PM;G34;1 afdemag schwehrThe answer is, "not very." That is not surprising. Saturation is somewhere way over 180mT and I probably saturnated (SIRM) with the 1T.
bp04-6gw-s1-094.0 10.0 0.4 2.210E-3 357.0 -2.1 3/21/05;3:13 PM;G34;1 irm schwehr x-in coil2
bp04-6gw-s1-094.0 5.0 0.4 1.983E-3 357.3 -1.6 3/21/05;3:18 PM;G34;1 afdemag-irm schwehr bp04-6gw-s1-094.0 7.5 0.4 1.866E-3 357.5 -1.7 3/21/05;3:20 PM;G34;1 afdemag-irm schwehr bp04-6gw-s1-094.0 10.0 0.4 1.763E-3 357.4 -1.7 3/21/05;3:21 PM;G34;1 afdemag-irm schwehr bp04-6gw-s1-094.0 15.0 0.4 1.588E-3 357.5 -1.6 3/21/05;3:23 PM;G34;1 afdemag-irm schwehr bp04-6gw-s1-094.0 20.0 0.4 1.437E-3 357.5 -1.8 3/21/05;3:24 PM;G34;1 afdemag-irm schwehr bp04-6gw-s1-094.0 30.0 0.4 1.195E-3 357.6 -1.8 3/21/05;3:26 PM;G34;1 afdemag-irm schwehr bp04-6gw-s1-094.0 40.0 0.4 1.006E-3 357.8 -1.9 3/21/05;3:27 PM;G34;1 afdemag-irm schwehr bp04-6gw-s1-094.0 50.0 0.4 8.581E-4 357.6 -1.9 3/21/05;3:29 PM;G34;1 afdemag-irm schwehr bp04-6gw-s1-094.0 60.0 0.4 7.409E-4 357.5 -2.1 3/21/05;3:30 PM;G34;1 afdemag-irm schwehr bp04-6gw-s1-094.0 80.0 0.4 5.720E-4 357.5 -2.0 3/21/05;3:32 PM;G34;1 afdemag-irm schwehr bp04-6gw-s1-094.0 100.0 0.4 4.577E-4 357.5 -2.3 3/21/05;3:33 PM;G34;1 afdemag-irm schwehr bp04-6gw-s1-094.0 120.0 0.4 3.763E-4 357.6 -2.1 3/21/05;3:35 PM;G34;1 afdemag-irm schwehr bp04-6gw-s1-094.0 150.0 0.4 2.915E-4 357.3 -2.3 3/21/05;3:36 PM;G34;1 afdemag-irm schwehr bp04-6gw-s1-094.0 180.0 0.4 2.320E-4 357.5 -2.2 3/21/05;3:38 PM;G34;1 afdemag-irm schwehr
03.22.2005 09:49
Mac stuff
Bailey is getting his first haircut since October this morning. He
was quite scruffy!
From Mac OSX Hints:
MySQL Administrator
From Mac OSX Hints:
This is simple but handy command to add LoginItems (items that are executed after the user logs in) via the command line using the default tool. To add a normal startup item, use this syntax:
defaults write loginwindow AutoLaunchedApplicationDictionary \ -array-add '' Hide Path \ /path/to/item
Of course, replace /path/to/item with the absolute path to whatever it is you want to open. Also, if you want the opened thing to hide itself when it opens, just change the word false, after the Hide key, to true. That's all there is to it.
Caution: If you form this command incorrectly, it's possible you could overwrite all of your login items -- backup the loginwindow.plist file before starting -- it's in your user's Library/Preferences folder
MySQL Administrator
03.22.2005 06:49
ISIS Update
So things are looking up for the next Mars landers having ISIS in
fink.
I thought I'd add some comments on this thread. First off we haven't made an official port of Isis 3 to MacOS X and the post by Kris is in regards to Isis 2. We are planning to have a fully native port for Isis 3 completed before October 2005. In terms of dependencies, we use:
qt image magick xerces xalan doxygen tnt/jama camgraph naif toolkit (cspice only) gmake g++/gcc rsync
Isis 3 does not have any proprietary software so that shouldn't be a concern and we hope to eliminate camgraph and xerces by using equivalent functions in qt (although I can't say when this will happen).
Since I'm Mac-illiterate I can't really weigh in on fink, although I suspect we will not use it in the port. However, to me it sounds like all you wanted was a network accessable tarball of our source tree. This is something that might be easily done on our part. I'm a little leary because we do have a required data area and it is quite large. I think to house both rsync'able and tarballs of the same data might be questionable.
This could be addressed in the make system for Isis 3. I'm hoping that in the future, we can integrate the selection and installation of the data (base system and mission specific) through make. This, along with the tarball of the source, which includes the make files, might be just what you need to build your fink files.
In your case, fink would grab the source tarball and execute the make command to build Isis 3 binaries and download the base data area. Then allow the user to choose which mission-specific data areas to make.
If this scenario might work for you please let us know.
Cheers, Jeff Anderson
03.21.2005 20:38
Lots of pmag and a little python
PyCon 2005
Papers. Like I have time to read any of these. Some interesting
titles:
Agile Testing with Python Test Frameworks matplotlib - from brain surgery to rocket science Scripting the Mac with Python pyblosxom: A microkernel approach to blogging A Python Preprocessing Framework for Hydrologic ModelingEnough python. Today was another full day of working on BPSIO-04 core data. I started off by collecting some samples in plasitic cylinders, which is what all those pictures are from. Here are the samples that I took:
bp04-5gw-s2-024 bp04-5gw-s2-028 bp04-5gw-s2-034 bp04-5gw-s2-124When it came to using the AGFM, I had some naming trouble that is now fixed up. The original names were:
bp04-5gw-s1-001 bp04-5gw-s1-022 bp04-5gw-s1-039
bp04-1gw-s2-035 bp04-1gw-s2-042 bp04-1gw-s2-081 bp04-1gw-s2-121
bp04-1gw-s1-002 bp04-1gw-s1-020 bp04-1gw-s1-035
12077 21 Mar 12:09 1W1-002.RAW 12080 21 Mar 12:09 1W1-081A.RAW 12077 21 Mar 12:09 1W1-121.RAW 12087 21 Mar 12:09 1W1-121B.RAW 5129 21 Mar 12:09 1W1-121C.RAW 12074 21 Mar 12:09 NI-2-1.RAWI renamed these to the following with core sections fixed:
2005-Mar-21-NI-2-1.raw bp04-1gw-s1-002.raw bp04-1gw-s2-081.raw bp04-1gw-s2-121-a.raw bp04-1gw-s2-121-b.raw bp04-1gw-s2-121-c.rawI do not really know how to deal with the calibrations, since it will be hard to give unique names by hand, so I guess I will use the date. I need to lookup the header field names. Here is what the calibrartion sample header looks like:
"Model 2900 ASCII Data File",7,-1,+2.000000E-01,+5.400327E+02, 2, +0.000000E+00, +1.000000E+02, +1.000000E+04, +1.000000E+04 , +0.000000E+00, 0, 202, 404, +2.000000E-03, 16, +8.952734E+01, +5.400000E+02, -8.497925E+01, +2.475232E-01, +5.563772E+01, 3, "ni-2 -1", +0.000000E+00, +0.000000E+00, +0.000000E+00, -1.000000E+03, "03/21/2005 11:00", 0, 0, 0, 0Which is then followed by:
+9.992000E+03,+1.756200E-03 +9.975000E+03,+1.755900E-03 +9.887500E+03,+1.756500E-03 +9.788000E+03,+1.756800E-03 +9.688000E+03,+1.757600E-03By the way, I took the extra close up pictures of 1GW-S1 for a reason. At 13-16cm you can see by eye that the material is mostly made up of fecal pellets. It does not seem to have worked accept for the right hand bottom corner of this picture:
P3210028.JPGJust read Lisa's 1995 paper of pseudo-Thellier:
Tauxe, L., Pick, T., and Kok, Y.S., Relative paleointensity in sediments: a pseudo-Thellier approach, Geophys. Res. Lett., 22, 2885-2888, 1995.
03.21.2005 10:08
Sampling for AGFM and Currie Balance
I have the majority of the AF demag work done. Today I sampled
plastic cylinders fro AGFM, Currie Balance, and smear slides:
21-Mar-2005 Sampling
03.20.2005 19:42
FreeVMS
FreeVMS
just had a new release. Now I have the potential of bad flashbacks to
working on hal and gal.arc.nasa.gov. My first and only experience on
VMS. VMS was the home to a lot of interesting ideas that still have
not made their way to other OSes. Plus the VAX instruction set was
crazy huge.
03.20.2005 09:37
switch statements in python
Last night, I was going back through blog entries trying to find
something that I had written recently for my thesis. I never found
it, but I did find quite a few typos. Ouch. And I did not even fix
them.
Wow. Google now has a python interface called goopy
I am having trouble plotting variable number of data streams with python gnuplot. So I am making a hack to handle 1..6 data objects. I wanted a switch statement, but was surprised to find none. But I did run into this switch blog article:
Wow. Google now has a python interface called goopy
I am having trouble plotting variable number of data streams with python gnuplot. So I am making a hack to handle 1..6 data objects. I wanted a switch statement, but was surprised to find none. But I did run into this switch blog article:
This works with lambdas as well, for inline calculations. Here's a switch statement that assigns to a variable in PHP:
switch ($value) { case 'a': $result = $x * 5; break; case 'b': $result = $x + 7; break; case 'c': $result = $x - 2; break; }
And here's the equivalent code in Python:
result = { 'a': lambda x: x * 5, 'b': lambda x: x + 7, 'c': lambda x: x - 2 }[value](x)
03.19.2005 18:11
CMU Atacama robot makes Slashdot
Saw a slashdot article today on CMU's Zoe in the Atacama Desert of
Chilean high desert.
Autonomous Robot Finds Life in Atacama Desert
The FRC Zoe web page
Autonomous Robot Finds Life in Atacama Desert
The FRC Zoe web page
03.19.2005 11:26
cwp-su 38 now in fink
Version 38 of cwp-su is now in fink unstable. This is still not
perfect. Here are some of my notes about the rough edges:
I am not able to build the Mesa GL GLUT based packages do to a clashing symbol, but hopefully I have fixed the problem for those that had undefined ICE symbols in xgraph.
psimage, pscontour are still _psimage, _pscontour in /sw/bin to avoid clashing with GMT. You will find them without the leading _ in ${CWPROOT}
The include files and static libs are not installed, so if I you write software against seismic unix, you are still out of luck.
I am not able to build the Mesa GL GLUT based packages do to a clashing symbol, but hopefully I have fixed the problem for those that had undefined ICE symbols in xgraph.
psimage, pscontour are still _psimage, _pscontour in /sw/bin to avoid clashing with GMT. You will find them without the leading _ in ${CWPROOT}
The include files and static libs are not installed, so if I you write software against seismic unix, you are still out of luck.
cc -I/Users/schwehr/Desktop/su/src/../include -I/Users/schwehr/Desktop/su/src/../include/MGL -I/usr/X11R6/include -I/usr/X11R6/include -O3 -Wall -ansi -pedantic -D_POSIX_SOURCE -DCWP_BIG_ENDIAN trip.c -L/Users/schwehr/Desktop/su/src/../lib -L/usr/X11R6/lib -L/usr/X11R6/lib -lglut -lXmu -lXt -lSM -lICE -lXext -lXi -lX11 -lm -lGL -lGLU -lglut -ltrack -lpar -lcwp -lm -o /Users/schwehr/Desktop/su/src/../bin/trip -I/sw/include -L/sw/lib ld: multiple definitions of symbol _err /usr/lib/libm.dylib(err.So) definition of _err /Users/schwehr/Desktop/su/src/../lib/libpar.a(errpkge.o) definition of _err in section (__TEXT,__text) ld: multiple definitions of symbol _warn /usr/lib/libm.dylib(err.So) definition of _warn /Users/schwehr/Desktop/su/src/../lib/libpar.a(errpkge.o) definition of _warn in section (__TEXT,__text)and if I remove the -lm, it will conflict with libSystem (which also provides the equiv to libm)
cc -I/Users/schwehr/Desktop/su/src/../include -I/Users/schwehr/Desktop/su/src/../include/MGL -I/usr/X11R6/include -O3 -Wall -ansi -pedantic -D_POSIX_SOURCE -DCWP_BIG_ENDIAN trip.c -L/Users/schwehr/Desktop/su/src/../lib -L/usr/X11R6/lib -L/usr/X11R6/lib -lglut -lXmu -lXt -lSM -lICE -lXext -lXi -lX11 -lGL -lGLU -lglut -ltrack -lpar -lcwp -o /Users/schwehr/Desktop/su/src/../bin/trip -I/sw/include -L/sw/lib ld: warning multiple definitions of symbol _err /Users/schwehr/Desktop/su/src/../lib/libpar.a(errpkge.o) definition of _err in section (__TEXT,__text) /usr/lib/libSystem.dylib(err.So) definition of _err ld: warning multiple definitions of symbol _warn /Users/schwehr/Desktop/su/src/../lib/libpar.a(errpkge.o) definition of _warn in section (__TEXT,__text) /usr/lib/libSystem.dylib(err.So) definition of _warn
03.19.2005 08:25
Fink and Scientific Programming/DataProcessing
I added a couple more books to my very draft book called Introduction
to Scientific Programming. Specifically, the Fish Book, The
Purple Book, and Visual Quickstart Python. If only all books were the
price of the Visual Quickstart series!
Working on getting cwp-su in fink upgraded to version 38. This was requested by Neal. I will try to get more of the X11 programs working at the same time. I forgot how long it takes to build lesstif even on a G5.
On the never ending todo list of things to add to fink: BHP_SU for seismic data cubes in SU. There is also I_SU which is another Tcl/Tk graphical frontend to CWP/SU.
Working on getting cwp-su in fink upgraded to version 38. This was requested by Neal. I will try to get more of the X11 programs working at the same time. I forgot how long it takes to build lesstif even on a G5.
On the never ending todo list of things to add to fink: BHP_SU for seismic data cubes in SU. There is also I_SU which is another Tcl/Tk graphical frontend to CWP/SU.
03.18.2005 08:48
USGS ISIS
I posted a message to the USGS ISUS support board. Here are the
messages from the thread.
I am looking into adding ISIS to fink (Mac OSX packaging tool) before the next Mars lander mission. The fink policy is to only allow systems that can be built from source. Unfortunetely, the rsync distribution mechanism makes in nearly impossible for fink to grab a snapshot of ISIS. I would have to rsync the source myself and create my own snapshot (which I would prefer not to do). Is there anyway to make available on the web a gzip or bzip2 compressed tar archive of upcoming releases in addition to rsync? Then fink could then doThe response
curl http://someusgsurl.usgs.gov/src/isis-3.1.0.tar.bz2
This tar would just be all the sources and support files excluding the very large mission datasets.
My goal with fink is to help create a complete planetary science workstation. Any person on the science team can get a Mac laptop and have as many of the mission tools as possible. This greatly reduces the load on the flightops support staff and puts more power in the hands of the science team members. I have also been trying to accomplish the same for marine geology/shipboard operations.
See this page for some of the software I have been packaging:
Maintainer=Kurt Schwehr
Thanks for you consideration, -Kurt Schwehr
Hi Kurt...My followup
Adding ISIS to Fink is not feasible at the moment for the following reasons:
ISIS is built entirely independant of the Fink environment. This is a concious decision we made because we felt the Fink overhead was significant and unnecessary (and we still feel that way).
There is some proprietary code used in ISIS, albeit a very small number of routines, but proprietary none-the-less.
The effort required to make ISIS Finkable would be in addition to our already preferred distribution method (rsync) and probably substantial in comparison to our current development environment (you do realize you are asking us to migrate development, or at least support, of Mac ISIS to the Fink environment).
We have 3rd party software dependancies for the Mac that exist only in binary form. We would like to take advantage of the availability of binary versions of software where ever possible to minimize support. These are, but not limited to:
OpenMotif A package called PGPLOT GNU Fortran Compiler (g77)
We have limited support staff and resources to support ISIS, let alone support the Mac OS X platform.
Fink is a great tool to migrate Unix/Linux software to the Mac platform, but the downside is that Fink software cannot run independant of that environment. Our goal is to provide ISIS to the widest community possible and minimize maintainence and installation difficulties.
In my opinion, DarwinPorts (http://darwinports.opendarwin.org/) provides more flexible (runs natively in the Mac environment) and appealing (does not require the enduser to install anything) ISIS software development support. With this approach, our ISIS developers can install the necessary 3rd party software ISIS depends on and only those software tools. We can then provide these dependancies (i.e., shared libraries) with the ISIS distribution. This scenario is obviously directly in conflict with the Fink policy as you state.
At this time, we do have a consistant development and distribution configuration for all operating systems we support (Linux, Solaris and Mac OS X). In my opinion, adding Fink to this mix creates additional overhead we cannot at this time support. _________________ Kris Becker Computer Scientist Astrogeology Team U.S. Geological Survey
Hi Kris,We will see how this goes. Plenty of time to work through this before mission time!
Thanks for your reply.
Here are some added some clarifications to what I meant in that last post.
* First: The last thing I want to do is add more work to your load. I am offering to do the leg work for an info file.
* 2nd: I am thinking long term here. I personally can not start working on this until I finish my PhD later this year. I ask that you consider allowing me to work on a fink info file sometime over the next two years before the 2007 mission.
* This does not have to impact your current development and release mechanism. Fink packages generally use what ever a project releases. Changing the development methods for such a large project would be not a good idea.
* I would be the one creating an info file to fink. That info file would take your source base as is. It is then up to me to create any patches and run any commands against that source base necessary to compile the program.
* I was thinking that you could add a tar cfj isis-3.0.3.tar.bz ISIS_ROOT to your release process and maybe add an exclude list of the proprietary files so that tar skips them. Then, maybe we could get community support to create replacements for those missing functions and APIs if they are as few as you suggest. This could be done without involving the USGS.
* Alternatively, I could create a fink info file that lives outside of the normal fink process that users place in their /sw/fink/10.3/local/main/sci directory. Then issues of licenses and proprietary code could be reduced.
* As for dependencies, that should not be a problem. Fink has info files for all of what you mentioned (g77, openmotif, and pgplot). The fink community can work on getting them into the stable tree where the servers build binaries for them.
* During MER operations, I noticed that there was a huge proportion Mac Laptops and many of those had fink. There were a few G5 Macs in flight ops this time and I think there will be many more Mac workstation setups for the 2007 mission.
* Fink and darwinports do have some collaboration between projects. They definitely have a different style, but in the science teams I am around, the science end users seem to really prefer fink with the FinkCommander program to provide functionality over the base Mac OSX system.
* Here is the complete list of current packages in fink:
http://fink.sourceforge.net/pdb/list.php
And links to the ones you mentioned.
http://fink.sourceforge.net/pdb/package.php/g77 http://fink.sourceforge.net/pdb/package.php/openmotif3 http://fink.sourceforge.net/pdb/package.php/pgplot
Please take your time thinking about this. I am not in any rush. Just wanted to plant the idea in your head. Feel free to ask me or anyone on the fink lists if you have any questions or thoughts.
Sincerely, Kurt Schwehr
03.17.2005 23:24
AF demag continues
The AF demag sessions continue. My list-missing.py program says that
I have 12 samples that not been through AF demag. The last sample
tonight was CRAZY!
Jason has got the blog tools working on sorcerer, but it is strange that I can not click on the title and get the entry to pop up in a web browser. But today marks the first serious lab entry documenting a liquid helium fill for BUBBA. I got into the lab just a few minutes after the fill was totally done, so I missed the action.
Check out this picture from Spirit on mars.
Jason has got the blog tools working on sorcerer, but it is strange that I can not click on the title and get the entry to pop up in a web browser. But today marks the first serious lab entry documenting a liquid helium fill for BUBBA. I got into the lab just a few minutes after the fill was totally done, so I missed the action.
Check out this picture from Spirit on mars.
03.16.2005 15:13
Spring AGU
For Spring AGU, I will be session Co-Chair with Laurie Brown for the
Monday poster session entitled, "Dynamos, reversals and much more."
03.16.2005 14:22
Senate Votes to Open Alaskan Oil Drilling
What a waste! If we ever open this area up to drilling, it should be
at least 10-20 years in the future when oil and gas is much more
expensive and environmental protection will be a much smaller fraction
of the cost and probably much better. This stuff will help the US
economy way more down the road compaired to now. How about massive
funding for conservation and alternatives to oil and gas? And please
do not say the solution is a "hydrogen economy." That is like saying
we will run on batteries. Hydrogen is just a storage means and a very
unstable one at that.
Senate Votes to Open Alaskan Oil Drilling
Senate Votes to Open Alaskan Oil Drilling
03.16.2005 12:14
MER discussion forum
Here is another public MER
discussion forum. Your mileage may vary.
03.16.2005 09:28
Disk mounting and Tracking mouse idle time
This was suggested by biovizier. UPDATE: It works!. Awesome.
sudo defaults write \ /Library/Preferences/SystemConfiguration/autodiskmount \ AutomountDisksWithoutUserLogin -bool trueAha! Now I got a post that goes into a little more detail. I think the above solution is better than hand editing the xml prefence file. 10.3: A fix for local drives not mounting until user login
Try this! It tells you how long the keyboard/mouse have been idle. I have never heardof /usr/sbin/ioreg before and the output from this program is strange, but the perl here cleans it up. Found this on mac osx hints:
sleep 5; ioreg -c IOHIDSystem | \ perl -ane 'if (/Idle/) \ {$idle=int((pop @F)/1e9); print $idle,"\n"; last}'Run it and then do not touch anything until it returns.
03.16.2005 08:50
JWZ picture tile
JWZ comes up with some pretty cools hacks. I like this one called picturetile.
This is basically what I was planning to do with pdflib today. Maybe
this can save me a huge about out time. I still want my afdemag down
core plots to have a better pattern.
UPDATE... How do I get this thing to work? Craps out on me with Mac OSX 10.3.8 complaining about now being able to allocate the size string in the ImageMagick perl module.
UPDATE... How do I get this thing to work? Craps out on me with Mac OSX 10.3.8 complaining about now being able to allocate the size string in the ImageMagick perl module.
03.15.2005 19:41
The AF Demag continues
Just checked the last two days' AF demag samples into cvs. I am down
to 59 samples left out of 257.
03.15.2005 11:08
Butterflies and RAW mode images
From Peter Franks:
Now I understand about the RAW format. It does make sense that the camera does not really have the computational power to compress frames and it would take a while. Nobody wants to wait a long time between being able to take the next frame. I bet over time RAW mode goes away as it becomes easier (smaller, less power, less $$$) to put compute power in a camera.
I'm pretty sure these are Painted Lady butterflies (Vanessa cardui). They are very widespread around the world (not found in Antarctica or Australia). They can form large populations in Mexico that then migrate north in the spring. We had a big migration some years back (probably 8 or so) - lots of guts on car windshields, etc. Seems to me it was the same year we had really spectacular bioluminescent red tides...And another take from Lanna Cheng:
We looked up Guide to Butterflies of California and think that they are probably Vanassa annnabella, a race of Californian Painted Lady. They are not Monarchs.I need to play with SoVRMLBillboard in Coin. Deanne Tucker made something like this in 1996-7 that worked well in straight SGI Open Inventor.
Now I understand about the RAW format. It does make sense that the camera does not really have the computational power to compress frames and it would take a while. Nobody wants to wait a long time between being able to take the next frame. I bet over time RAW mode goes away as it becomes easier (smaller, less power, less $$$) to put compute power in a camera.
lrosenstein on Tue, Mar 15 '05 at 01:54PM
RAW format captures all the information the camera has without any processing. Most cameras only capture one color channel per pixel. Producing even a 16-bit TIFF from that would require some processing in the camera.
You are right about the flexibility of TIFF, and in fact Adobe's new digital negative format is based on the TIFF standard.
03.15.2005 10:14
MER, landslides, and more
First, lots of pictures to share of the Landslides at SIO, Fisheries, and Blacks Beach.
Jason's Becca's Kurt'sI am glad this is NOT the view right outside my office door! Is is sliding? I am not an engineering geologist, but this does not look good. This ground is not very competent.
2M160631572 from Mars. Is this a blueberry that has been ratted or something else? Or was this just brushed. Looks like a light ratting.
Check out the pmag blog started by Jason.
Plans Calendar looks like a nice alternative to phpicalendar.
This morning was the day of the butterflies as I drove right above the Acquarium around 9:30 AM. There were hundreds of them.
Maybe someone will tell me that raw is supposed to be better than 16 bit per channel tiffs.
03.14.2005 11:18
python Gnuplot
I am now into the swing of things with gnuplot-py. First, aquaterm is
started by gnuplotx. That is how gnuplot is showing the results with
the default terminal. Then my big bug from yesterday was why my lists
were all corrupted. Here is what NOT to do when you initialize
lists. This is what I assumed would work from my C/C++ background:
depth = tau1 = tau2 = tau3 = []Turns out that this makes four references to ONE list. Whoops. Here is the working version. It makes a png image of the graph. You will not be able to reproduce this until I publish and release the data, but the concept will hopefully be a good template for those needing a jumpstart.
#!/usr/bin/env python import os # os.remove() import sys # sys.exit() import string import sqlite # pysqlite import bpsio import Gnuplot
def plot_susc(db_cx, filename, table, corenum, maxdepth): cu = db_cx.cursor() searchStr = "SELECT depth,tau1,tau2,tau3 FROM " + table searchStr += " WHERE corenum=" + str(corenum) + " ORDER BY depth;" cu.execute(searchStr)
depth = []; tau1 = []; tau2 = []; tau3 = [] for row in cu.fetchall(): depth.append(row['depth']) tau1.append(row['tau1']) tau2.append(row['tau2']) tau3.append(row['tau3']) t1 = Gnuplot.Data(tau1,depth,with='lp') t2 = Gnuplot.Data(tau2,depth,with='lp') t3 = Gnuplot.Data(tau3,depth,with='lp') g = Gnuplot.Gnuplot()
title = table + " core " + str(corenum) + " AMS tau123" g.title(title) g.xlabel("Tau (Sums to 1)") g.ylabel("Depth(cm)") g("set yrange [0:"+str(maxdepth)+"] reverse") g("set size ratio 2") g("set output '"+filename+"'") g("set terminal png size 320,480") g("set xtics 0.01") g.plot(t1,t2,t3)
if __name__ == '__main__': cx = sqlite.connect (bpsio.dbFileName); plot_susc(cx,"foo.png","ams_geo",1,200)
03.13.2005 13:36
USGS ISUS
Add this to the list of things to fink-a-fy:
USGS ISUS (Integrated Software for Imagers and Spectrometers) is now available via rsync.
Maybe Vicar will follow along too?
Something keeps starting AquaTerm. What is doing this?
I posted to the system forum of mac osx hints hoping for help on Volume unmounting at logout. the thread. No replies yet.
USGS ISUS (Integrated Software for Imagers and Spectrometers) is now available via rsync.
Maybe Vicar will follow along too?
Something keeps starting AquaTerm. What is doing this?
I posted to the system forum of mac osx hints hoping for help on Volume unmounting at logout. the thread. No replies yet.
03.13.2005 13:11
Distinctive Image Features from Scale-Invariant Keypoints
Saw this paper via slashdot which makes big claims about this for
tying different images together.
Python Robotics project: pyro
PythonMagick - native image magick python interface.
OpenQVIS - Volumetric program on top of Coin3D. Different than SIMVoleon. Is this still in active development?
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, International Journal of Computer Vision, 60, 2 (2004), pp. 91-110.
Abstract This paper presents a method for extracting distinctive invariant features from images, which can be used to perform reliable matching between different images of an object or scene. The features are invariant to image scale and rotation, and are shown to provide robust matching across a a substantial range of affine distortion, addition of noise, change in 3D viewpoint, and change in illumination. The features are highly distinctive, in the sense that a single feature can be correctly matched with high probability against a large database of features from many images. This paper also describes an approach to using these features for object recognition. The recognition proceeds by matching individual features to a database of features from known objects using a fast nearest-neighbor algorithm, followed by a Hough transform to identify clusters belonging to a single object, and finally performing verification through least-squares solution for consistent pose parameters. This approach to recognition can robustly identify objects among clutter and occlusion while achieving near real-time performance.
Python Robotics project: pyro
PythonMagick - native image magick python interface.
OpenQVIS - Volumetric program on top of Coin3D. Different than SIMVoleon. Is this still in active development?
03.12.2005 23:40
Using PowerBook with the lid closed
I was looking for a quick fix to the problem that Jason discovered
(firewire drives unmount where there is no user logged in) and ran
into directions on how to use your Powerbook G4 with the lid closed.
But on closer examination, this is NOT what I hoped. How about a
powerbook as a headless server?
How to use your PowerBook G4 with the display closed
03.12.2005 22:46
Searching for seafloor eruption
http://www.uwnews.org/article.asp?articleID=8815
University of WashingtonMaybe this is another good place to grab basalitic glass for paleointensity ground truth experiments. What we really need is insitu magnetic field readings right at the time of eruption directly at the site of cooling.
The most intense swarms of earthquakes detected in the last 10 to 12 years on the far edge of the Juan de Fuca plate could indicate the eruption of magma from the seafloor or an underwater volcano. Between 50 and 70 earthquakes an hour, most of them small, were occurring at the end of February at a spot some 200 miles off the Canadian coast. ...
03.12.2005 15:24
ivcon and mars dust devil
Wow! Dust
devil on Mars!! This is really cool.
I'm looking at ivcon again to see if it can solve some of the problems I have been having. The main problem is lots of autogenerated files in OpenInventor that must make their way to Lightwave, Blender, and Maya for visualization. DeepExploration works, but is Windows only, costs, and it very slow to use. Also, models do not get an intelligent name so people have to load each model and rename it. Not good! Animators have better things to do with there time.
I got a little bit of a start in that Burkardt's web page moved since the last time a looked. Here it is:
I was not able to get the version on Burkardt's web page to build on Mac OSX. It builds fine on Linux, but it has huge static arrays on the stack that upset the Mac OSX tool chain. I was able to build the version from sf.net/projects/ivcon. All it needs is to comment out the INFINITY definition in mesh.cpp and the addition of a Makefile like this one.
I'm looking at ivcon again to see if it can solve some of the problems I have been having. The main problem is lots of autogenerated files in OpenInventor that must make their way to Lightwave, Blender, and Maya for visualization. DeepExploration works, but is Windows only, costs, and it very slow to use. Also, models do not get an intelligent name so people have to load each model and rename it. Not good! Animators have better things to do with there time.
I got a little bit of a start in that Burkardt's web page moved since the last time a looked. Here it is:
IVCON - 3D Graphics File ConversionMy goal is to generate quality Alias Wavefront OBJ files that can be quickly animated into a nice scene with as little effort as possible.
I was not able to get the version on Burkardt's web page to build on Mac OSX. It builds fine on Linux, but it has huge static arrays on the stack that upset the Mac OSX tool chain. I was able to build the version from sf.net/projects/ivcon. All it needs is to comment out the INFINITY definition in mesh.cpp and the addition of a Makefile like this one.
SRCS:=asc.cpp SRCS+=ase.cpp SRCS+=byu.cpp SRCS+=dxf.cpp SRCS+=gmod.cpp SRCS+=hrc.cpp SRCS+=iv.cpp SRCS+=ivconv.cpp SRCS+=main.cpp SRCS+=obj.cpp SRCS+=pov.cpp SRCS+=smf.cpp SRCS+=stl.cpp SRCS+=tds.cpp SRCS+=tec.cpp SRCS+=tmat.cpp SRCS+=tri.cpp SRCS+=txt.cpp SRCS+=ucd.cpp SRCS+=utils.cpp SRCS+=vla.cpp SRCS+=wrl.cpp SRCS+=xgl.cpp OBJS:=${SRCS:.cpp=.o} TARGETS:=ivconv mesh CXXFLAGS:= -O3 -funroll-loops -fexpensive-optimizations -DNDEBUG -ffast-math -mpowerpc targets: ${TARGETS} ivconv: ${OBJS} g++ $^ -o $@ ${CXXFLAGS} mesh: mesh.o g++ $^ -o $@ ${CXXFLAGS} clean: rm ${OBJS} ${TARGETS}That is all well and good until I tried to give it one of the OpenInventor mesh files. Doh!
ivconv foo.wrl foo.objHere are some extracts from foo.wrl/foo.iv:
initializing...Bad data VertexProperty Bad data texCoord Bad data normalIndex Bad data textureCoordIndex Bad data -1 DATA_REPORT - The input file contains: Bad data items 5 Text lines 142514 Text bytes (binary data) 0 Colors 0 Comments 1 Duplicate points 0 Faces 0 Groups 0 Vertices per face, maximum 0 Line items 0 Points 19748 Objects 0 DATA_CHECK - Data checked. EDGE_NULL_DELETE: There are a total of 0 edges. Of these, 0 were of zero length, and deleted. FACE_AREA_SET: Minimum face area is 0.000000 Maximum face area is 0.000000 FACE_NULL_DELETE There are a total of 0 faces. Of these, 0 passed the order test. COR3_RANGE - Data range: Minimum Average Maximum Range X -15.038561 -6.743375 -2.956082 12.082479 Y 1.490623 3.606597 8.242425 6.751802 Z -0.457958 1.089934 2.803719 3.261677 OBJ_WRITE - Wrote 19754 text lines.
#Inventor V2.1 asciiWhat if I ivfix or ivToVRML the file? Sent the file to an SGI and created foo.wrl. The results are a bit better but not perfect. I now get faces in addition to vertices. This is no good if the texture does not come along too! Here is what ivconv says:
Separator {
Texture2 { filename "foo.png" } ShapeHints { vertexOrdering COUNTERCLOCKWISE shapeType UNKNOWN_SHAPE_TYPE creaseAngle 1.5 } Material { ambientColor 1 1 1 diffuseColor 1 1 1 specularColor 0 0 0 emissiveColor 0 0 0 shininess 0 transparency 0 } IndexedTriangleStripSet { vertexProperty VertexProperty { vertex [ -9.5078192 4.1908059 0.080789998, -9.5088491 4.2086911 0.096816003, -9.515564 4.1935291 0.096675001, ... -3.3741801 1.669412 0.46248901, -3.377382 1.663372 0.46251801, -3.3802321 1.657196 0.462392, -3.3816769 1.650483 0.46164301, -3.383647 1.643977 0.461128 ] normal [ 0.84313971 -0.27994934 0.45906833, 0.76509589 -0.38959566 0.51268256, 0.78889918 -0.27429348 0.54991013, 0.78889918 -0.27429342 0.54991019, 0.83235496 -0.18904907 0.52100438, ... 0.31228593 0.19732746 -0.92926818, 0.53384936 -0.030668387 -0.84502321, 0.26931736 -0.21443249 -0.93887532, 0.53384936 -0.030668389 -0.84502321, 0.57270825 -0.1091051 -0.8124662 ] texCoord [ 0.78613299 0.97070301, 0.78125 0.96582001, 0.78613299 0.96582001, 0.79101598 0.96582001, 0.38085899 0.96582001, ... 0.81054699 0.013672, 0.81542999 0.013672, 0.82031202 0.013672 ] orderedRGBA [ ] normalBinding PER_VERTEX_INDEXED materialBinding OVERALL
} coordIndex [ 19747, 19691, 19746, -1, 19691, 19690, 19746, -1, 19745, 19689, 19744, -1, 19689, 19688, 19744, -1, 19744, 19687, 19743, -1, 19743, 19687, 19686, -1, 19743, 19686, 19742, -1, 19741, 19685, 19684, -1, 19741, 19684, 19740, -1, 19740, 19684, 19683, -1, 19740, 19683, 19739, -1, 19739, 19683, 19682, -1, 19737, 19681, 19736, -1, 19681, 19680, 19736, -1, 19736, 19680, 19679, -1, 19736, 19679, 19735, -1, ... 40, 39, -1, 39, 38, 36, 37, -1, 28, 27, 26, 25, -1, 19, 17, 18, 16, -1, 16, 17, 15, -1, 14, 13, 12, 11, -1, 12, 11, 10, 9, -1, 9, 7, 8, 6, -1 ] normalIndex [ 68580, 68578, 68579, -1, 68394, 68576, 68577, -1, 68572, 68388, 68571, -1, 68387, 68570, 68569, -1, 68568, 68378, 68567, -1, 68566, 68376, 68565, -1, 68564, 68372, 68563, -1, 68558, 68559, 68560, -1, 68558, 68556, 68557, -1, 68553, 68554, 68555, -1, 68553, 68552, 68551, -1, 68551, 68550, 68357, -1, 68546, 68353, 68545, -1, 68353, 68543, 68544, -1, ... 45, 43, 44, 41, -1, 28, 25, 27, 26, -1, 26, 25, 21, -1, 20, 18, 19, 15, -1, 17, 15, 16, 13, -1, 12, 9, 11, 10, -1 ] textureCoordIndex -1
} }
ivconv foo.wrl foo.obj initializing... DATA_REPORT - The input file contains:Where is the texture. Guess I am going to have to write something myself.
Bad data items 0 Text lines 163654 Text bytes (binary data) 0 Colors 0 Comments 1 Duplicate points 0 Faces 37019 Groups 0 Vertices per face, maximum 3 Line items 0 Points 19748 Objects 0
03.12.2005 14:57
Blacks Beach slide
Here is a photo by Becca taken earlier today of the Blacks Beach slide!
Becca's pic
03.12.2005 12:37
SPIE Visualization and Data Analysis pubs
Here are by two publications from earlier this year:
Visualization tools facilitate geological investigations of Mars Exploration Rover landing sitesAnd the other:
Kurt D. Schwehr, Carrie Nishimura, Catherine L. Johnson, Debi L. Kilb, Atul Nayak
Publication:
Proc. SPIE Vol. 5669, p. 135-145, Visualization and Data Analysis 2005; Robert F. Erbacher, Jonathan C. Roberts, Matti T. Gröhn, Katy Börner; Eds.
Publication Date: Mar 2005
Designing visualization software for ships and robotic vehicles
Kurt D. Schwehr, Alexander Derbes, Laurence Edwards, Laurent Nguyen, Eric Zbinden
Publication:
Proc. SPIE Vol. 5669, p. 226-237, Visualization and Data Analysis 2005; Robert F. Erbacher, Jonathan C. Roberts, Matti T. Gröhn, Katy Börner; Eds.
Publication Date: Mar 2005
03.12.2005 12:18
iGMT and Sutar Box Coring
iGMT -
an interactive interface (Tcl/Tk based) for Generic Mapping Tools
(GMT). This is another thing on my fink todo list. I wonder if iGMT
will work with GMT 4.0? The download link seems to be broken.... but
this one works:
ZigBee is another X10 like standard for home automation. Someday I will have time to play with all these things.
Doh. The latest template-pm581 does not build on fink for me.
Just ran into these notes on how to use the Sutar Box Corer on the Sproul from the Sept 2003 cruise off of San Onofre. Chief Scientist was Babcock.
Proceedure for using a Sutar Box Core
http://www.seismology.harvard.edu/~becker/igmt/igmt_v1.2-20040303.tar.gzAlso on the fink/port todo list is OpenEV, a GIS type library. Also Mapserver, OGR, OGDI.
ZigBee is another X10 like standard for home automation. Someday I will have time to play with all these things.
Doh. The latest template-pm581 does not build on fink for me.
Just ran into these notes on how to use the Sutar Box Corer on the Sproul from the Sept 2003 cruise off of San Onofre. Chief Scientist was Babcock.
Proceedure for using a Sutar Box Core
- Put out two i bolts on the deck far enough apart to fit the whole box core frame.
- Transfer to under the A-Frame
- Strap it down with ratchet strap.
- Set the top door with pins
- Tie the ropes from bottom doors to the loops with light string
- Pull surgical tubing over trigger
- Attach a frame winch to red O link using a shackle
- Attach tag lines through the two rope loops attached to handles and clip one end to the aft cleets of the A-Frame
- Remove rachet strap.
- (Leave the life lines in place -- back railing)
- Lift box core up so that the weights are up and the core is off the deck
- Pull the two flagged safety pines
- A-frame out
- lower winch
- When box corer is in the water, remove tag lines.
- Record GPS time and location
- lower box core to 50 meters above the sea floor at 25 m/sec
- then 20 m/sec to the bottom
- let out 3 extra meters of line to let the core get into the mud.
- record GPS time and location
- life to 50 m/sec below the surface
- up to the top at 15 m/s
- Use two recovery poles to grab the two garden hose covered loops.
- Pull the lines into the ship and ditch the recovery poles.
- Raise the box core up and over the life lines using the crane and the a-frame at the same time.
- Place the box core just above the deck.
- Put the two safety pins in the sides.
- Then lower the frame onto the deck.
- Secure it with a rachet strap and remove the winch wire.
- Secure the winch wire.
- Lift up the top door and secure it with the pin.
- Siphon the water off with some flexible tubing. Need 6 feet of tubing to do this easily.
- (FIX: insert tube prep info above.)
- Push core liner tubes in the mud, down to the bottom.
03.11.2005 07:47
Black Beach landslide
Cliff
Collapses Near Blacks Beach (NBC San Diego)
Officials with the San Diego Fire Department told NBC 7/39 that lifeguards and a city geologist had been called to the scene after the hillside gave way about ½ mile north of Scripps Pier.
03.10.2005 08:38
updated info for gmtplus
Just updated the install info for gmtplus.
For the few Biologists out there who have examined fungal initials being formed on solid media. Please tell me what is wrong, purely on the visual evidence, not your belief that conditions on Mars cannot allow life to exist on the surface, with the interpretation that clear indications are evident in this picture that the blueberries are being formed from the aggregation of large cell like structures in the ground matrix. Here are some notes about fink-a-frying a new 10.3.8 box. Woo hoo 800MHz G4. Ouch that is slow! But slow and steady wins the race. There will be a blend of apt-get and fink for installing as I try to jump start this as fast as possible. Some of these programs take hours to build. If you have the time, use "fink install" and not apt-get install to get the latest stuff since you will have to rebuild much anyways down the road.
My gmtplus web pageApple has a web page on installing and running X11. It even talks through fink! Cool. Just avoid using xhost if you can. Much better to tunnel X11 over ssh with the -X ssh option.
Runnin X11Just read this over on a public mars rover discussion. It made me smile. That is the nice thing about public discussions. I would just dismiss the possibility that these things are alive. But it never hurts to play what if. This could be the basis for a nice SciFi story Live Blueberries
For the few Biologists out there who have examined fungal initials being formed on solid media. Please tell me what is wrong, purely on the visual evidence, not your belief that conditions on Mars cannot allow life to exist on the surface, with the interpretation that clear indications are evident in this picture that the blueberries are being formed from the aggregation of large cell like structures in the ground matrix. Here are some notes about fink-a-frying a new 10.3.8 box. Woo hoo 800MHz G4. Ouch that is slow! But slow and steady wins the race. There will be a blend of apt-get and fink for installing as I try to jump start this as fast as possible. Some of these programs take hours to build. If you have the time, use "fink install" and not apt-get install to get the latest stuff since you will have to rebuild much anyways down the road.
sudo emacs /sw/etc/fink.conf # Add unstable/main and unstable/crypto to Trees fink index fink selfupdate sudo apt-get install emacs21 gnuplot gv wget-ssl doxygen gsl gengetopt imagemagick sudo apt-get install qt3 fink install postgresql73-ssl python24 python24-socket-ssl scietificpython-py24 sqlite fink install density # This is a couple hours fink install postgresql73-ssl fink install sqlite fink install python24 fink install python fink install pysqlite-py24 fink install sqlite3 fink install biggles-py24 f2py-py24 gnuplot-py24 numarray-py24 fink install proj fink install gsl fink install gdal fink install mbsystem fink install pmag fink install tksu # you will have to answer some questions for it to build. #fink install weka # Errr... this is currently broken. Sorry. fink install xtermcontrol g3data fink install pdflib-py24 fink install density soxt3 fink install scipy-py24 scientificpython-py24 fink install pygsl-py24 fink install tksu # brings cwp-su (seismic unix) fink install twisted-py24 fink install wxpython-py24 fink install pyxml-py24 fink install rpy-py24 fink install octave # Very much link matlab fink install gdl # Very much like IDL fink install matplotlib-py24 fink install ipython-py24 fink install povray graphvizThis was hobbled together so it may not be perfect. There are many packages which can not be installed in the same install line due to different versions of ncurses dev and readline dev that have to be temporarily installed. You are on your own to get sioseis and pltsegy since they are not in fink.
# finish up... fink clean fink scanpackages
03.09.2005 18:50
Hoekstra Lab
I ran into Hopi Hoekstra today on upper campus. I did not know that
there was anyone else from my high school class at UCSD. Hopi is a
faculty member in the biology department: Hopi
Hoekstra
03.09.2005 08:56
MER driving record
177 meters is a long way for a little rover to drive. I guess that is
the nice thing about the Opportunity landing site. When not in a
crater, it can drive pretty much as far as the power constraints will
allow. Here is the article: Mars
Rovers Break Driving Records, Examine Salty Soil Hey MWM,
congrats on the new software! We did it with Nomad back in 1997, but
this is the real deal!
A nicely written article by Bio-ITworld: Perfect : Revenge is sweet for Apple, now savoring the rise of Mac OS X and serve in the lab, but this is still the planting. A difficult growing season lies ahead. BY JOHN RUSSELL. Only bummer is that it does not talk about all the goodness that can come from using fink or darwinports for open source software.
The bright patch of disturbed soil, dubbed "Paso Robles," has the highest salt concentration of any rock or soil ever examined on Mars. Combined information gained from inspecting it with Spirit's three spectrometers and panoramic camera suggests its main ingredient is an iron sulfate salt with water molecules bound into the mineral. The soil patch is also rich in phosphorus, but not otherwise like a high-phosphorus rock, called "Wishstone," that Spirit examined in December. "We're still trying to work out what this means, but clearly, with this much salt around, water had a hand here," Squyres said.This next one is for Jason... Set up Airport Express and third-party wireless . How to jump on the bandwagon with your neighbors. If you have an open access point, I hope you can fend off the RIAA and prove it was not you who bootlegged all the Willy Nelson music...
A nicely written article by Bio-ITworld: Perfect : Revenge is sweet for Apple, now savoring the rise of Mac OS X and serve in the lab, but this is still the planting. A difficult growing season lies ahead. BY JOHN RUSSELL. Only bummer is that it does not talk about all the goodness that can come from using fink or darwinports for open source software.
03.08.2005 19:17
Mt St Helens erupts
Local NBC coverage of Mount
St. Helens erupting again. I still remember May 18th, 1980.
03.08.2005 15:50
gnuplot-py to pdf
I now know that pdf output works just fine. That was my last major
concern as I can pass what ever to gnuplot. I was worried for a few
that gnuplot-py would override the terminal or output setting.
#!/usr/bin/env python import Gnuplot g = Gnuplot.Gnuplot()I noticed that gnuplot-py uses HappyDoc instead of pydoc. Pydoc has been frustrating me a bit. It is great. Don't get me wrong. Doxygen has some features, like explicitly calling out what each arg passed to a function is for and complains if I do not document them. I want that incessant nagging that makes sure I doc all of my interface args! Will HappyDoc do this? Is is really "Happy?"
x = [1,2,3,4,5] y = [1,2,2,3,2.5] gData = Gnuplot.Data(x,y,with='lp') g('set terminal pdf') g("set output 'foo.pdf'") g.plot(gData)
03.08.2005 15:19
Super simple gnuplot-py example
My first ever gnuplot-py program:
#!/usr/bin/env python import GnuplotOkay, so this is nothing special and I have been using gnuplot extensively while working on my thesis. Why this is exciting to me is that I was swimming in a pool of little space delimited ascii data files. There are 10's of little routines running around my thesis cvs tree that just read some sort of database and turn around to write little text files. There there are maybe 50 gnuplot files that try to plot all this. I constant do "make clean" do nuke all of them and I have come up with lots of silly extensions for them...
g = Gnuplot.Gnuplot() g.title('Plot 1') g("set xlabel 'yo x label'")
x = [1,2,3,4,5] y = [1,2,2,3,2.5] gData = Gnuplot.Data(x,y,with='lp') g.plot(gData)
.ddi --- depth dec inc .di --- dec inc .dt --- depth and a text string t123 --- for AMS Tau 1,2, and 3 f12-f23 --- F test susc --- Bulk susceptibility etc.This is getty pretty darn messy. I would rather just have the dumped pdf plots. Gnuplot-py will now make these plots without every leaving any of these droppings on my filesystem. I am starting to think about xcore-py which would make generating inventor models a snap.
03.08.2005 12:28
truetype fonts work in pil
Well, I was right. Jeff W. had a quick fix for the Python Imaging
Library (pil). I now have a working test case for loading a true type
font. It is a bit of a weird one since it is the first thing I ran
into. I am trying to get as much done as possible before I talk to
Lisa and Neal tomorrow afternoon. First, here is how I randomly found
the font I used in the example:
Thanks Jeff!!
locate .ttf | grep '\.ttf$' | lessI used a random font from Apple's X11. This font is not pretty, but it works.
#!/usr/bin/env python import Image, ImageDraw, ImageFont im = Image.new("RGB",(400,400),"white") draw = ImageDraw.Draw(im)By the way, I have PIL 1.1.4-26 installed. This will not work with older versions of PIL from fink.
font = ImageFont.truetype("/usr/X11R6/lib/X11/fonts/TTF/luxirb.ttf",30)
draw.text((10,100),"hello", font=font, fill="red") im.show()
Thanks Jeff!!
03.08.2005 11:04
Wrapping a core in a 3D environment
I heard a rumor yesterday that IVS will soon have a core photo on a
cylinder feature in Fledermaus. This is my take on that from 2003 in
the Visualization Class.
Thanks to Jill W. for the dataset, which is the resistivity log from an ODP hole off the coast of Oregon. She was using a Schlumberger tool to look for sinusoids in the data that would be cracks. You will notice that she found one in the brown layer in the foreground.
Thanks to Jill W. for the dataset, which is the resistivity log from an ODP hole off the coast of Oregon. She was using a Schlumberger tool to look for sinusoids in the data that would be cracks. You will notice that she found one in the brown layer in the foreground.
03.08.2005 10:10
Coin Designer 1.0
This is in the "I should really try this and put it in fink category."
Version 1.0 of CoinDesigner has been released.
Carbonate chimney in the dep ocean: Hydrogen And Methane Sustain Unusual Life At Sea Floor's 'Lost City'
View Google Map and TerraServer image for one location
Version 1.0 of CoinDesigner has been released.
Carbonate chimney in the dep ocean: Hydrogen And Methane Sustain Unusual Life At Sea Floor's 'Lost City'
View Google Map and TerraServer image for one location
03.08.2005 07:52
Mac laptop accelerometer
This is cool!
Powerbook sudden motion sensor and then bumptoons. I am bummed that it appears my 1.5 GHz laptop does not have a motion sensor.
Duke, Woods Hole Geologists Discover 'Clockwork' Motion By Ocean Floor Microplates
Powerbook sudden motion sensor and then bumptoons. I am bummed that it appears my 1.5 GHz laptop does not have a motion sensor.
Duke, Woods Hole Geologists Discover 'Clockwork' Motion By Ocean Floor Microplates
03.06.2005 14:33
afdemag components
I have being trying to think about how to handle components that I
pick from the AF demag data that I have been collecting. My first
thought was a table that has each sample as a row and fill in as many
components as I need up to some maximum with each component typed to
be fit with a line, plane, or both. A line is just a directional
point on a stereonet, while a plane is an arc on a stereonet. The
following samples are hypothetical.
Then comes the issue of how to get this data into my sqlite database. I am leaning towards a new style that uses python code to store the data in a list. If I keep it simple, I will be readable by other programming languages with a little bit of parsing, but not too much. If I specify both the index and the treatment, then hopefully I can double check and catch typos. Maybe something like this:
UPDATE:
Looking at Lisa's zeqplot_magic.py, it looks like the names for the fields above are:
s = start, Indx = index, Trmt = treatment levelYou can see from the example above that it is possible to have overlapping definitions. However, it is hard to parse this table and I am limited by the number of components that I want to look at. The alternative is to waste a lot of space and allow many components. Instead, I am liking an alternative treatment table:
samplename type1 s1Indx s1Trmt e1Indx e1Trmt type2 s2Indx s2Trmt e2Indx e2Trmt
1gws2-030 line 1 5.0 4 15.0 plane 5 20 9 60 1gws2-033 both 0 0.0 5 20.0 line 3 10 10 80
samplename trmtIdNum type startIndx startTrmt endIndx endTrmt 1gws2-030 0 line 1 5.0 4 15.0 1gws2-030 1 plane 5 20.0 9 60.0 1gws2-030 2 line 0 0.0 -1 -1.0The last line would be fit a line (aka direction) to all of the samples from sample 1gws2-030.
Then comes the issue of how to get this data into my sqlite database. I am leaning towards a new style that uses python code to store the data in a list. If I keep it simple, I will be readable by other programming languages with a little bit of parsing, but not too much. If I specify both the index and the treatment, then hopefully I can double check and catch typos. Maybe something like this:
tKeys=('er_specimen_name','trmtIdNum','type','startIndx','startTrmt','endIndx','endTrmt'); t=[] t.append(('1gws2-030', 0, 'line', 1, 5.0, 4, 15.0)) t.append(('1gws2-030', 1, 'plane', 5, 20.0, 9, 60.0)) t.append(('1gws2-030', 2, 'line', 0, 0.0, -1, -1.0))Note that I did not use the full sample name convention for my samples. I would normally call 1gws2-030
bp04-1gw-s2-030
UPDATE:
Looking at Lisa's zeqplot_magic.py, it looks like the names for the fields above are:
measurement_step_min measurement_step_max specimen_calculation_type = l or p (but I will make them line or plane) er_specimen_name specimen_comp_name
03.06.2005 13:04
sci-oppix
I have been thinking about the best way to handle the paleomag code
for the whole community. The big problem is that many people in our
community insist in using MS-Windows. Windows has piddly squat for
package management. It is not a nice thing to tell someone that they
need to install cygwin and make sure to get X11 and Fortran. Then
they need to also get python and sqlite correctly installed. Once
that is done, they need to install all the pmag software. If they are
doing stuff like I do, they also need coin, soqt/sowin, SimVoleon,
gmt, and mbsystem. Ouch. That can be rough on an experienced person
if the Windows box is not in perfect condition. Plus uninstalling
software on ms-windows always leaves behind some moose-poop in the
registry.
Is there an easy answer? Well here is an answer that is easy on the end user but hard on us maintainers. We need to create a derivative of knoppix call sci-oppix or pmagoppix. The idea is that we build some custom debian packages for all the all these new items. Then we can remove some of the large packages from knoppix that we use and insert ours. These end users just put the bootable sci-oppix cd in their windows computer and reboot. It comes up with a GUI that takes them through importing, processing, and submitting to MagIC all of their mag data. It can read/write data from the NTFS partitions, USB drives, or other attached firewire drives. Then, when they are done, they reboot to Windows. Some users might not even realize that they are running Linux or have a clue what Linux, GNU, or Debian are. Now I just need a whole pile of free time devoted to this
While we are at it, we could build versions for x86, sparc, and ppc/mac of these bootable CDs. Even mac users could just use the CD. This is a way to guarantee that the required software and libraries are present and working.
Is there an easy answer? Well here is an answer that is easy on the end user but hard on us maintainers. We need to create a derivative of knoppix call sci-oppix or pmagoppix. The idea is that we build some custom debian packages for all the all these new items. Then we can remove some of the large packages from knoppix that we use and insert ours. These end users just put the bootable sci-oppix cd in their windows computer and reboot. It comes up with a GUI that takes them through importing, processing, and submitting to MagIC all of their mag data. It can read/write data from the NTFS partitions, USB drives, or other attached firewire drives. Then, when they are done, they reboot to Windows. Some users might not even realize that they are running Linux or have a clue what Linux, GNU, or Debian are. Now I just need a whole pile of free time devoted to this
While we are at it, we could build versions for x86, sparc, and ppc/mac of these bootable CDs. Even mac users could just use the CD. This is a way to guarantee that the required software and libraries are present and working.
03.06.2005 11:08
More measuring
Another day of measuring afdemag on flo.
Last night I got to talk to Kevin Xu a little bit. He is visiting for the week to work on some of the PNG cores.
So my 2nd python test case for the sql code from yesterday showed me that I had a fundamental design flaw. Dictionaries do not keep their order. So if I want to write test cases with a dictionary, I need to know the dictionary order for the expected string. I guess I could switch over to lists of tuples, but there is nothing wrong with the dictionary. Is the sort order of a dictionary stable? It makes sense that it world be as python probably keeps the keys sorted. On the other hand, there is no reason that this sorted alphanumerically. It could be a hash function. If it is, Guido and company could change which has function they are using which would make my test cases very unstable. And now that I look at my test code results, it is not alphanumeric forward.
Last night I got to talk to Kevin Xu a little bit. He is visiting for the week to work on some of the PNG cores.
So my 2nd python test case for the sql code from yesterday showed me that I had a fundamental design flaw. Dictionaries do not keep their order. So if I want to write test cases with a dictionary, I need to know the dictionary order for the expected string. I guess I could switch over to lists of tuples, but there is nothing wrong with the dictionary. Is the sort order of a dictionary stable? It makes sense that it world be as python probably keeps the keys sorted. On the other hand, there is no reason that this sorted alphanumerically. It could be a hash function. If it is, Guido and company could change which has function they are using which would make my test cases very unstable. And now that I look at my test code results, it is not alphanumeric forward.
Which gave this as the created string:
def testSqlInsertStrFromDictTwo(self): """Try two values""" aDict = {'one':1,'two':2.} expectedResult="insert into myTable (one,two) values (1,2.0);" got=sqlInsertStrFromDict("myTable",aDict) print got self.failUnless(expectedResult==got)
insert into myTable (two,one) values (2.0,1);Looks like lists work just fine! Here is my current code that passes the 3 embedded test cases. It still needs lots more test cases and maybe I will get lucky and it will work if I throw a dictionary instead of a list as the input pairs. After a quick test, it does!
#!/usr/bin/env python """LIST VERSION"""Back to measuring...
import unittest
def sqlInsertStrFromList (table,aList): """ Take a list and make an insert string""" ins = "insert into " + table + " (" first=True for pair in aList: key = pair[0] if first: first=False else: ins+="," ins+=str(key) ins += ") values (" first=True for pair in aList: value = pair[1] if first: first=False else: ins+="," ins+=str(value) ins += ");" return ins
class SqlTests(unittest.TestCase): def testSqlInsertStrFromListOne(self): """The simplest case I can come up with. Table field called 'one' with value of the number 1""" aList = [('one',1)] expectedResult="insert into myTable (one) values (1);" got=sqlInsertStrFromList("myTable",aList) #print got self.failUnless(expectedResult==got)
def testSqlInsertStrFromListFloat(self): """Try a float""" aList = [('two',2.)] expectedResult="insert into myTable (two) values (2.0);" got=sqlInsertStrFromList("myTable",aList) #print got self.failUnless(expectedResult==got)
def testSqlInsertStrFromListTwo(self): """Try two values""" aList = [('one',1),('two',2.)] expectedResult="insert into myTable (one,two) values (1,2.0);" got=sqlInsertStrFromList("myTable",aList) #print got self.failUnless(expectedResult==got)
def testSqlInsertStrFromDictOne(self): """Will a dict work too? It seems to!""" aDict = [('one',1)] expectedResult="insert into myTable (one) values (1);" got=sqlInsertStrFromList("myTable",aDict) #print "dict one: ", got self.failUnless(expectedResult==got)
def main(): unittest.main()
if __name__ == '__main__': main()
03.06.2005 05:57
VTP - Virtual Terrain Project
Got my yearly ping from the Global Terrain
NASA GLOBE Data 30 Arc Sec database but supposed to be better than GTOPO30.
Damn... googlebot went crazy this morning. It checked out 2322 files, but did not download anything since the total KBytes was 2! Whacky. I guess that is just making sure the google cache is all up to date.
NASA GLOBE Data 30 Arc Sec database but supposed to be better than GTOPO30.
Damn... googlebot went crazy this morning. It checked out 2322 files, but did not download anything since the total KBytes was 2! Whacky. I guess that is just making sure the google cache is all up to date.
03.05.2005 22:32
quick ssh command tutorial
If you want to ssh to another command and run something on the remote
machine (which is useful for e.g. a backup script) here are some
examples to get you going:
# The simplest example... localhost is the same as your current machine ssh localhost lsParty on Wayne. Party on Garth.
# now make it a remote machine ssh pixie ls
# if you are having name resolution troubles where it says # "who the heck is pixie" ssh pixie.ucsd.edu ls
# You need quotes if the command had any spaces ssh pixie "ls -l"
# What if you need to switch users? ssh smith@pixie "ls -la"
# What if you are not sure that you are getting to the right host? ssh pixie hostname ssh pixie "uname -a" ssh pixie pwd
# So you want to backup the etc directory on a remote machine to a # tar file and copy it to your current computer? ssh smith@pixie "tar cf etc.tar /etc" ssh smith@pixie "bzip2 -9 etc.tar" scp smith@pixie:etc.tar.bz2 . tar tfjv etc.tar.bz2
03.05.2005 17:50
py unittest first time
I just wrote my first little python tidbit using unit test. I want a
function that takes a dictionary and builds the insert query. This
makes building a database much easier. Maybe pysqlite does this, but
I could not figure out how. It is great having the unit tests. Much
cleaner than what I usually do in C++. Here is my initial code now
that it passes the most basic test case:
#!/usr/bin/env python
def sqlInsertStrFromDict (table,aDict): """ Take a dict and make an insert string"""
ins = "insert into " + table + " (" first=True for key in aDict.keys(): if first: first=False else: ins+="," ins+=str(key) ins += ") values (" first=True for value in aDict.values(): if first: first=False else: ins+="," ins+=str(value) ins += ");" return ins
import unittest class SqlTests(unittest.TestCase): def testSqlInsertStrFromDictOne(self): aDict = {'one':1} expectedResult="insert into myTable (one) values (1);" got=sqlInsertStrFromDict("myTable",aDict) print got self.failUnless(expectedResult==got)
def main(): unittest.main()
if __name__ == '__main__': main()
03.05.2005 11:34
Octave tutorial part 2
Octave tutorial
part 2 for Octave which is
mostly compatible with Matlab.
03.05.2005 09:17
K15 or Kn in python rmag_ams.py
I took an hour and put together what I think the interface should be
like for ams data in python. Comments? Anything is open to change at
this point. Is rmag_ams.py the right filename?
rmag_ams.html
I was thinking that it would work something like this:
rmag_ams.html
I was thinking that it would work something like this:
import rmag_ams file = rmag_ams.K15file() file.open("antarctica.k15","r") # Write out all the sample entries file.writeMagIC("antartica.samples.magic")Alternatively, it could start out like this, which would allow different numbers of positions...
# get one sample sample = file.getNextK15()
new = rmag_ams.K15() new.sampleholder = 4.75 new.K[0] = 1.077E+2 new.K[1] = 1.057E+2 .... new.K[13] = 1.076E+2 new.K[14] = 1.071E+2
new.write("measurements.k15")
file = rmag_ams.KnFile(numPos=9)
03.05.2005 07:16
MER instrument swap
NewScientist article by Jenny Hogan (there was a different person from NewScientist at JPL for the first 5 months)
"Now the reason for this is obvious," Gellert told (New Scientist. "We found proof that we unintentionally swapped both instruments." Spirit is carrying the spectrometer destined for Opportunity, while Opportunity is in possession of Spirit's.Interesting little story by Aglssis on slashdot. Is this true? I have not heard about this one, so I can not vouch for it being true.
Small problems lead to medium sized problems which lead to big problems. Example: In the 1970's the NRC was similar to the Department of Transportation or FAA (pre 9/11) in that their job was to help facilitate the nuclear economy, not to beat down offenders. In the early 70's plant managers at a nuclear power plant in Alabamba, Browns Ferry Nuclear Power Plant, received reports that their insulation connecting to a cable room was not in accordance with fire specifications (small problem). Since this was not a significant problem, managers ignored it. Later workers testing the air-tightness of the room failed to follow the correct procedures by using candles to check the air tightness (if the flame is deflected, air is moving in that direction--small problem). Managers were aware but dismissed the problem. During testing for air leaks the flame of a candle was sucked into insulation and a fire erupted. The cable run that caught on fire was non-redundant and carried all of the control features for two nuclear reactors. Control of the reactors was lost and reactor safety was severly compromised. Problems that occurred included that the operators of the reactors did not know how to properly respond to this causality (including attempts to put out a large class A fire with portable CO2 extinguishers). Over $100 million in damages occurred, but the reactors narrowly escaped tragedy (medium sized problem). This occurred in 1975 and the NRC mostly covered up the problem. No congressional hearing were held. No significant corrective actions were issued and review of the ability of the operators to fight a causality at a nuclear power plant was not reviewed. Fast forward four years and we arrive at Three Mile Island (big problem), where many of the shortcomings of the Brown's Ferry Plant and of the NRC being able to regulate and control the nuclear industry were exposed.Google weather forecast:
La JollaAre these both by Amazon? Interesting searches of my web server:
http://a9.com/schwehr.org http://www.alexa.com/data/details/?url=schwehr.orgYou can review a web site on Alexa. That is kind of a funny concept. Maybe I should fake a glowing review
03.04.2005 17:43
busy day
I saw the PNG cores today as almost the whole crew was up at the core
locker processing. Interesting to see how others go about it. Also,
was talking to Dingler about digital cameras since Warren showed some
really awesome digital core photos. Woo hoo! I am still totally
confused about why RAW images would be any better than say a 16-bit
per channel lossless tiff. It would be easy to store just raw dn
levels in the tiff plus add tags or sub images for all the camera data
and even bad pixels / dark frame for correction later on. Someone
please set me strait!
Yesterday, I talked to Lisa about MagIC for AMS/K15 kappabridge lowfield data. I am starting to come up with an idea for what I think the python API should be. However, I am starting to question the idea of the intermediate magic text files. It does not hurt to also be able to read and write them, but use them? I can just make an sqlite database and we could bridge over to the official magic db when you submit. No worrying about delimiters etc... parsing is pretty much a non issue except that many fields are a check of text with many subfields separated by ":". That could be a little interesting. Anyways, here are my initial thoughts on APIs:
I have also been thinking about a wrapper around the database so that if I change the backed database, then I am ok.
Yesterday, I talked to Lisa about MagIC for AMS/K15 kappabridge lowfield data. I am starting to come up with an idea for what I think the python API should be. However, I am starting to question the idea of the intermediate magic text files. It does not hurt to also be able to read and write them, but use them? I can just make an sqlite database and we could bridge over to the official magic db when you submit. No worrying about delimiters etc... parsing is pretty much a non issue except that many fields are a check of text with many subfields separated by ":". That could be a little interesting. Anyways, here are my initial thoughts on APIs:
class K15file: open(filename,"rw") close() eof() """Are we at the end?""" getNextK15() """Return a K15 class filled in with next value""" appendK15(k15record) """Add an ams measurement to end of the file""" writeMagic(filename) """Write out the k15 file to the magic format""" # for writeMagic, might want 2 file names, one for samples, one # for other params # Do we need a class for all the other fields that go along? # maybe just keep it simple and this writes the samples file?
class K15: setFrom4Lines(str1,str2,str3,str4) setFrom1Line() setFromDatabase(cx,samplename) set(samplename, yadayada, susceptabilities[15]) addToDB(cx) getHext() """Return a hext class""" getS() """ Return a list of 7 values """ getParamter(paramName) """returns F,L,P', etc"""
I have also been thinking about a wrapper around the database so that if I change the backed database, then I am ok.
03.04.2005 10:24
more pil fonts
I got a font to load and work. This is not great since it does stuff
outside of fink that is whacky and non-standard. The key is that I
can move on with real work. First, I needed to download some
.pil/.pbm fonts. Then I have to add those to the python path.
Finally, a load_path command is used to snag them.
My .bashrc gets this
My .bashrc gets this
export PYTHONPATH=$PYTHONPATH:\ ${HOME}/projects/xcore/doc/BPSIO-04/work/pilfontsThen I need my makefile to ensure that the fonts are present.
pilfonts: pilfonts.zip unzip pilfonts.zip pilfonts.zip: wget http://effbot.org/downloads/pilfonts.zipFinally a sample python program that draws the font:
#!/usr/bin/env python import Image, ImageDraw, ImageFont im = Image.new("RGB",(400,400),"orange") draw = ImageDraw.Draw(im) draw.text((10,100),"default") font = ImageFont.load_path("helvR24.pil") draw.text((10,200),"helvR24", font=font, fill="red") im.show()This will show a tiny white default font with a larger 24 point font below in red all on an orange background.
03.04.2005 09:40
fonts with pil
The Python Imaging Library (pil) seems to be lacking in documenation
on how to load fonts in. I just want a big bold font to lay on png's
that are going into a poster via PDFlib lite. The first thing I ran
into on google groups was a suggestion to load a truetype font. So I
made a quick test program and kaboom!
Just noticed pygsl-py24 in fink... sweet! A wrapper for the GNU Scientific Library.
Some day I need to pull out the old jaz disks for Haughton and Yellowstone and try to recover all that great data with something like this:
#!/usr/bin/env python # ls /Library/Fonts/[a-zA-Z]*.ttfIt looks like the fink pil does not build the module _imagingft:
import Image, ImageDraw, ImageFont im = Image.new("RGB",(400,400),"white") draw = ImageDraw.Draw(im)
font = ImageFont.truetype("Raanana",30) draw.text((10,100),"hello", font=font, fill="red") im.show()
File "./fonttest.py", line 9, in ? font = ImageFont.truetype("Raanana",30) File "/sw/lib/python2.4/site-packages/PIL/ImageFont.py", line 189, in truetype return FreeTypeFont(filename, size, index) File "/sw/lib/python2.4/site-packages/PIL/ImageFont.py", line 116, in __init__ import _imagingft ImportError: No module named _imagingftDoh! I'm hoping Jeff W. will have a quick fix up his sleeve.
Just noticed pygsl-py24 in fink... sweet! A wrapper for the GNU Scientific Library.
Some day I need to pull out the old jaz disks for Haughton and Yellowstone and try to recover all that great data with something like this:
dd bs=512 if=/dev/rXX# of=/some_dir/foo.dmg conv=noerror,sync
03.03.2005 15:16
Bubba continued
I talked to Jason about Bubba today. Couple things were up. First
there was a good amount of grit at the bottom of the tube. The next
thing is that I had the sample holder offset off by about half a cm.
I was using the wrong part of of the line. The slider on the tower
goes BETWEEN the two lines. Then we concluded that the cube sample
holder is just not great. Comparing to Flo, we got the same and
intensity and inclination, but declination is just not going to be
good. So no more Bubba for me.
Talked to Lisa about the MagIC stuff. I am going to try to fold in the anisotropy part of the rock mag as I work on my thesis stuff. This is nice. I have been just creating my one table names as I go along and this way I will use what the folks at the IRM have voted to call all the fields and tables.
Talked to Lisa about the MagIC stuff. I am going to try to fold in the anisotropy part of the rock mag as I work on my thesis stuff. This is nice. I have been just creating my one table names as I go along and this way I will use what the folks at the IRM have voted to call all the fields and tables.
03.03.2005 07:44
unison
Unison (which is in fink) looks pretty cool. Lets you sync files both
ways between machines. Sounds like it may have some functionality
that is not available in rsync. Builds fine, but have not tried it.
03.03.2005 06:44
Bad Bubba, BAD
I tried to use the older CTF magnetometer (known as Bubba) last night
since it is right next to the SI-4 AF demagnetizer. Two hours later,
that was it. No more. BAD Bubba. It is not setup well for pmag
cubes, which have to SIT on a pedestal as they are lowered down to the
squids. I got CSDs that were like 3-5 which is not good. Plus the
boxes were having drift of up to 4-5 x 10-8. I must have been doing
something wrong. The AF demag plots looked like garbage. When I
switched back to using Flo (the 2-G magi), things looked much better.
The machined cube holder that Jeff and I had done a couple years ago
makes things much better. CSD's of like 0.7-1.0 until the sample is
really demagnetized.
"Squids measure Emus"
"Squids measure Emus"
03.02.2005 11:38
Mac Volume Viz
No idea if this would work with Coin/Voleon or how good it is. But it
is interesting news:
2. Big news for folks interested in using our VolumePro 1000 technology with a MAC! Antoine Rosset, Lead developer of Osirix software, signed an agreement with us at the beginning of the year where he will distribute and support the MAC drivers and Software that are compatible with VolumePro 1000. We will be responsible for selling the hardware. We have been shipping product to the MAC community through this relationship for over a month now and I am personally very excited about this new market opportunity! We are still working on putting together an announcement for our website. In the meantime, here is the link for the software and drivers:
osirix
03.02.2005 11:16
Another NASA MER site?
This is different than the marsrovers.jpl.nasa.gov. Not sure why
there are two, but here it is.
NASA MER Mission siteSomething funny from my MER pipeline document. I somehow felt it important to document the color of the mission op floors.
a = Spirit ( = 2 ) -- Maroon -- 4th floor b = Opportunity ( = 1 ) -- Blue -- 5th floor Teal -- 6th floor
03.02.2005 11:04
SGI raid troubles
Becca noticed that the console on the RAID on our SGI file server says
too many bad sectors on a drive and that it needs to be replaced.
Here is a quick excerpt of the /var/adm/SYSLOG... where the heck am I
supposed to be notified about these. Just on a little LCD display is
idiotic from a copmany that claimed it produces enterprise class
systems. I have removed duplicated messages of those that are not
relevant:
There is no hint in there that there is disk trouble. It is nice to see daily backups and it is amusing that the alias for that is named "croncrap@igpp". The is one of the reasons I never want to use an SGI again:
Just found out via IM that apparently this message has been there since November and that it is "ok" according to the admin. Note that this was found out word of mouth. There is no note on the machine. Supposedly because of the card that was bought for the raid. Time to throw the whole chain off the pier. I can't find squat about squat in the logs:
End grumpy mode
Feb 28 23:19:25 3D:stowe autofsd[264]: getmapent_uns: matlab5.2 automount:auto.net returned NS_NOTFOUND Feb 28 23:19:25 3D:stowe autofsd[264]: mount of /net/matlab5.2 failed Feb 28 23:19:41 6D:stowe inetd[325]: received SIGHUP: reconfiguring
Is this automated or was an SA on the system?
Feb 28 23:20:24 6C:stowe sendmail[312938]: j217KOqY312938: localhost.ucsd.edu [127.0.0.1] did not issue MAIL/EXPN/VRFY/ETRN during connection to MTA-v4 Feb 28 23:22:49 6A:stowe unix: pkgadjust: core file access error What is this?
Feb 28 23:23:38 6C:stowe sendmail[313858]: j217Nbud313858: from=root, size=453, class=0, nrcpts=1, msgid=<200503010723.j217Nbud313858@stowe.ucsd.edu>, relay=root@loca lhost Feb 28 23:23:38 6C:stowe sendmail[313713]: j217NcqY313713: from=, size=756, class=0, nrcpts=1, msgid=<200503010723.j217Nbud313858@stowe.ucsd.edu> , proto=ESMTP, daemon=MTA-v4, relay=localhost.ucsd.edu [127.0.0.1] Feb 28 23:23:38 6C:stowe sendmail[313858]: j217Nbud313858: to=croncrap@igpp.ucsd.edu, ctladdr=root (0/0), delay=00:00:01, xdelay=00:00:00, mailer=relay, pri=30066, re lay=localhost.ucsd.edu. [127.0.0.1], dsn=2.0.0, stat=Sent (j217NcqY313713 Message accepted for delivery) Feb 28 23:23:38 6C:stowe sendmail[313903]: j217NcqY313713: to= , ctladdr= (0/0), delay=00:00:00, xdelay=00:00:00, mailer=e smtp, pri=120369, relay=igpp.ucsd.edu. [132.239.152.7], dsn=2.0.0, stat=Sent (j217Ncar021999 Message accepted for delivery) Mar 1 01:02:14 6E:stowe rshd[314430]: root@sole.ucsd.edu as root: cmd='sh /export/stowe/BACKUP/stowe.day' Mar 1 12:53:22 4A:stowe unix: WARNING: ARP: got MAC address 0:3:93:6:c1:b2 on ef for BCAST IP address 0.0.0.0
... many complaints about matlab
Mar 1 23:33:24 3D:stowe autofsd[264]: getmapent_uns: matlab5.2 automount:auto.net returned NS_NOTFOUND Mar 1 23:33:24 3D:stowe autofsd[264]: mount of /net/matlab5.2 failed Mar 1 23:33:39 6D:stowe inetd[325]: received SIGHUP: reconfiguring
This has to be a cron thing
Mar 1 23:34:22 6C:stowe sendmail[320889]: j227YMqY320889: localhost.ucsd.edu [127.0.0.1] did not issue MAIL/EXPN/VRFY/ETRN during connection to MTA-v4 Mar 1 23:36:50 6A:stowe unix: pkgadjust: core file access error
Grr
Mar 1 23:37:44 6C:stowe sendmail[321070]: j227biHZ321070: from=root, size=603, class=0, nrcpts=1, msgid=<200503020737.j227biHZ321070@stowe.ucsd.edu>, relay=root@loca lhost Mar 1 23:37:44 6C:stowe sendmail[319122]: j227biqY319122: from=, size=904, class=0, nrcpts=1, msgid=<200503020737.j227biHZ321070@stowe.ucsd.edu> , proto=ESMTP, daemon=MTA-v4, relay=localhost.ucsd.edu [127.0.0.1] Mar 1 23:37:44 6C:stowe sendmail[321070]: j227biHZ321070: to=croncrap@igpp.ucsd.edu, ctladdr=root (0/0), delay=00:00:00, xdelay=00:00:00, mailer=relay, pri=30066, re lay=localhost.ucsd.edu. [127.0.0.1], dsn=2.0.0, stat=Sent (j227biqY319122 Message accepted for delivery) Mar 1 23:37:44 6C:stowe sendmail[320233]: j227biqY319122: to= , ctladdr= (0/0), delay=00:00:00, xdelay=00:00:00, mailer=e smtp, pri=120367, relay=igpp.ucsd.edu. [132.239.152.7], dsn=2.0.0, stat=Sent (j227biar001845 Message accepted for delivery) Mar 2 01:15:48 6E:stowe rshd[320770]: root@sole.ucsd.edu as root: cmd='sh /export/stowe/BACKUP/stowe.day' Mar 2 18:40:59 6E:stowe sshd[323119]: Accepted publickey for schwehr from 67.120.232.149 port 55834 ssh2
There is no hint in there that there is disk trouble. It is nice to see daily backups and it is amusing that the alias for that is named "croncrap@igpp". The is one of the reasons I never want to use an SGI again:
man -k raid No manual entries seem appropriate for raid.Wait, what is this file from August 2004?
# schwehr@stowe 12 $ cd /var/adm && locate RAID bash: locate: command not found
# schwehr@stowe 16 $ cd /var/adm && head 20040817.SYSLOG Aug 15 01:01:06 6F:stowe syslogd: restart Aug 15 01:01:35 6A:stowe unix: ql3d3: <6>Selection timeout Aug 15 01:01:35 6A:stowe unix: dksc3d3s7: SCSI driver error: device does not respond to selection Aug 15 01:01:35 1A:stowe unix: ALERT: I/O error in filesystem ("/stoweraid") meta-data dev 0xe2 block 0x40 ("xfs_trans_read_buf") Aug 15 01:02:05 6A:stowe unix: ql3d3: <6>Selection timeout Aug 15 01:02:05 6A:stowe unix: dksc3d3s7: SCSI driver error: device does not respond to selection Aug 15 01:02:05 1A:stowe unix: ALERT: I/O error in filesystem ("/stoweraid") meta-data dev 0xe2 block 0x40 ("xfs_trans_read_buf") Aug 15 01:02:35 6A:stowe unix: ql3d3: <6>Selection timeout Aug 15 01:02:35 6A:stowe unix: dksc3d3s7: SCSI driver error: device does not respond to selection Aug 15 01:02:35 1A:stowe unix: ALERT: I/O error in filesystem ("/stoweraid") meta-data dev 0xe2 block 0x40 ("xfs_trans_read_buf")Oh, that was AFTER the primary raid completely dead when a bad drive was not replaced right away. At that point it was too late to just swap the bad drive in and have a happy system rebuild the raid array.
Just found out via IM that apparently this message has been there since November and that it is "ok" according to the admin. Note that this was found out word of mouth. There is no note on the machine. Supposedly because of the card that was bought for the raid. Time to throw the whole chain off the pier. I can't find squat about squat in the logs:
egrep -i "raid|stowe|disk" [a-jl-x]* | egrep -v\ 'auto\.net|matlab|NOUSER|export|sendmail|password|Illegal\ user|SIGHUP|MAC|pkgadjust|freespace|account is locked|stack limit\ exceeded|startino|rc failed|identification string'\Blasted thing. I used to love IRIX compared to the other unix systems. I can figure out how to do all this with Linux or OSX raid arrays just fine.
oSYSLOG:Feb 20 01:01:06 6F:stowe syslogd: restart oSYSLOG:Feb 20 03:00:06 6B:stowe fsr[247588]: no rw efs file systems in mtab oSYSLOG:Feb 20 03:00:06 6B:stowe fsr[247543]: fsr_xfs -m /etc/mtab -t 7200 -f /var/tmp/.fsrlast_xfs ... oSYSLOG:Feb 20 05:29:37 6B:stowe fsr[248630]: fsr_xfs startpass 2, endpass 3, time 8971 seconds
End grumpy mode
03.01.2005 19:58
ACD rocks the global flyer
Check out Alex's screen shot: Global
Flyer Flight Path. He did this today with Coin and a gps feed.
Nice!
03.01.2005 13:14
Robot wars - Gunn High
Probably not the robot wars you were thinking about. Back in the day,
Roy, Ruwan and I beat the team from Gunn. They had a machine shop and
pretty metal parts. We had our garages and insane disregard for
anything remotely "normal." We were fast and perfect. They were slow
and almost perfect
AP-CA--RobotWars 02-28 0168 Recognized high school robotics program shut down amid threats
PALO ALTO, Calif. (AP) -- It looks like a case of geeks gone bad at Palo Alto's Gunn High School. The school's nationally recognized robotics program has been shut down amid accusations of bullying and threats among team members that prompted the departure of the team's coach. Bill Dunbar took a medical leave after parents and students complained he had ignored several disputes among team members, some of which ended up in court. One boy asked for a restraining order against a teammate he said had punched him. And a girl and her father requested a separate restraining order against a teammate they said used Internet instant messages and intimidating s`tares to pressure the girl into a romantic relationship. The Gunn robotics team won eight national championships in recent years and was once praised by a Forbes publication as a breeding ground for promising scientific leaders.
03.01.2005 10:25
pydoc rules!
Wow. pydoc is even better than doxygen! This is so simple and I
think it looks great. I just did this to learn about it:
python import pydoc help(pydoc)From there, I saw that there is a command line option to generate html file for modules:
pydoc -w build_db open build_db.htmlI need to figure out how to make the module links work right, but wow...
03.01.2005 09:39
Sumatra - Kerry Sieh
Topic: Sumatra Megathrust Earthquake
Yesterday we had a standing room only talk by Kerry Sieh from Caltech. He talked about his work there with both GPS stations, GPS campaign style, and coral micro-atols both pre and post the big Dec 2004 quake. The U-Th coral data from chainsawed sections is pretty cool. He was here before and his story just keeps getting more and more compelling. I especially liked the photo of the old rice patty that had been below sea level but was uplifted by the recent quake. Expect it to slowly subside for the next 200 years until the next big one. No one had remembered that rice patty in recent memory. I still have not found fledermaus files for any of those areas from the new cruises.
News bit: I hear rumors of a new 1 minute global bathymetry model in the Sandwell lab. My source says that abyssal hills are going to be visible in many areas. That will be cool!
Today's paleomag paper discussion will be on "Why are geomagnetic excursions not always recorded in sediments? Constraints from post-depositional remanent magnetization lock-in modeling, " Roberts and Winklhofer, EPSL, 227 (2004) 345-359. A few quotes:
Yesterday we had a standing room only talk by Kerry Sieh from Caltech. He talked about his work there with both GPS stations, GPS campaign style, and coral micro-atols both pre and post the big Dec 2004 quake. The U-Th coral data from chainsawed sections is pretty cool. He was here before and his story just keeps getting more and more compelling. I especially liked the photo of the old rice patty that had been below sea level but was uplifted by the recent quake. Expect it to slowly subside for the next 200 years until the next big one. No one had remembered that rice patty in recent memory. I still have not found fledermaus files for any of those areas from the new cruises.
News bit: I hear rumors of a new 1 minute global bathymetry model in the Sandwell lab. My source says that abyssal hills are going to be visible in many areas. That will be cool!
Today's paleomag paper discussion will be on "Why are geomagnetic excursions not always recorded in sediments? Constraints from post-depositional remanent magnetization lock-in modeling, " Roberts and Winklhofer, EPSL, 227 (2004) 345-359. A few quotes:
We assume that a PDRM can only start locking -in when substantial bioturbation ceases.They used the word nomogram which I found this definition. What a strange word! Isn't this just a graph?
... consolidation of ... sediments often proceeds in an exponential manner during initial stages.
... the change of documenting an excursion is greater for u-channel measurements compared to 2-cm discrete samples.
variations in sedimentation rates (up to a factor of 40 in extreme cases) in deep-sea drift deposits
[sedimentation] variation by a factor of 10 is common on glacial/interglacial timescales
polarity features will only be recorded if they last longer than the median lock-in depth divided by the sedimentation rate
a graphic representation of numerical relations
03.01.2005 08:04
March begins
February was the first month that I had a blog message for everyday of
the month. Okay, Feb is cheating, I know. Also, I don't know if that
is a good thing to shout about
Last night, Casey Luskin was on Fox 6 News San Diego talking about his belief in Intelligent Design, basically that a god entity had to have created some or all of what we see happening and that Darwinian evolution can't be what created the world we have today. Not my cup of tea. I am not sure how you would test this as a scientific theory. Their argument must be something like like the trouble of finding the lowest potential energy well in a space. If you roll a ball from the edge, there the deepest or "best" hole may never be reached. It takes outside intervention to find it.
Casey got his Masters in the SIO Paleomag studying the Snake River area with Lisa and Phil from Santa Barbara.
MacSlash has a discussion on attaching microscope cameras to your mac
Last night, Casey Luskin was on Fox 6 News San Diego talking about his belief in Intelligent Design, basically that a god entity had to have created some or all of what we see happening and that Darwinian evolution can't be what created the world we have today. Not my cup of tea. I am not sure how you would test this as a scientific theory. Their argument must be something like like the trouble of finding the lowest potential energy well in a space. If you roll a ball from the edge, there the deepest or "best" hole may never be reached. It takes outside intervention to find it.
Casey got his Masters in the SIO Paleomag studying the Snake River area with Lisa and Phil from Santa Barbara.
MacSlash has a discussion on attaching microscope cameras to your mac