Setting up Python-2.7 on CentOS 6

I just got a Linux desktop with CentOS 6 installed, and I need to set up the environment as Python-2.7 + Pandas + Numpy + Scipy with Emacs 24.3

It turns out that it's very tricky to use yum to install ANYTHING... (at least compared to Ubuntu)

First of all, install Python-2.7 side by side with the original 2.6 since otherwise you will mess up your OS. Then, install pip and configure it to Python-2.7. The step by step instruction is here:
https://github.com/0xdata/h2o/wiki/Installing-python-2.7-on-centos-6.3.-Follow-this-sequence-exactly-for-centos-machine-only

http://toomuchdata.com/2012/06/25/how-to-install-python-2-7-3-on-centos-6-2/

It's important to run:

yum groupinstall "Development tools"
yum install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel

Before compiling and installing Python.

Then, by typing:

pip install PACKAGE_NAME

You will be able to install the package for Python-2.7.

For Scipy, the important package to install before it are:
yum install blas blas-devel lapack lapack-devel atlas atlas-devel

For Matpoltlib, the important package to install before it are:
yum install freetype-devel libpng-devel

For iPython to have [Tab] nationalities:
pip install readline

To install Emacs 24:

http://vitalvastness.wordpress.com/2013/07/03/installing-emacs-24-on-centos-6/comment-page-1/

Install liblockfile from here (that’s the x86_64 link) … if you click on the download link it will invoke the package manager and install directly from Firefox.

cd /etc/yum.repos.d

wget http://pj.freefaculty.org/EL/pjku.repo

rpm –import http://pj.freefaculty.org/EL/PaulJohnson-BinaryPackageSigningKey

yum install emacs-24.2-4.el6.x86_64

Several Tips for Publishing Images from Matlab: PDF to Word, Font Size in Ubuntu etc.

Matlab provides some great convenience in creating high quality figures. However, esp. when you try to integrate these figures into your Word version of the paper, quite a few tweaks are needed.

1. Create, crop and insert high quality PDF images into word.

Matlab can create PDF or EPS figure quite easily by the print command, and you can even use LaTeX to combine multiple figures into one when you have to (subplot in Matlab create a very large margin between figures). However, one problem is to crop off the margins around the PDF. "pdfcrop" is a command line program in Ubuntu, but apparently it can not remove the margin on the bottom of the PDF. I found a nice Java program:

http://www.pdfscissors.com/

This can do the job quite nicely.

Then, before inserting PDF into word, keep in mind that there is no way to include such vector image into M$ Word documents. We have to convert it into high-quality TIFF image. There are quite a few options for this, just make sure to create TIFF image with 600 dpi. Just to be safe.

2. Change font size of Matlab figure in Ubuntu and save it into EPS.

This is a weird problem I had for a long time. Every time I tried to set the font size of Matlab figure in Ubuntu, it does not work. The same code works fine in Windows 7. The problem was that a font package is missing for Ubuntu. Just type:


sudo apt-get install xfonts-75dpi

Log out and log in, then it all works fine. Also, before save the figure as EPS, make sure to include:

set(gcf, 'PaperPositionMode', 'auto');

Find and replace a word in all the text document

So some day I need to replace a word to another one for all the XML file in a folder. I was like OK I'll write a C++ code. There goes another hour of my working time then.

Then I received this piece of code from a colleague:
find . -name "*.xml" -exec perl -pi.org -e 's/DICOMIO/CartoXP/' {} \;

Works like a charm.

Also, a linux command line to check how many cores do you have:
cat /proc/cpuinfo | grep processor | wc -l

Upload images to our BBS abroad

Seems that our BBS can not allow we to upload image from other countries...
OK. This script can bypass the IP checking and do the uploading. It is developed by Zimu Liu.

#!/bin/sh

if [ -z "$1" ]; then
 echo "Usage:  ./`basename \"$0\"` "
 exit -1
fi

FULLPATH="$1"
if [ ! -r "$FULLPATH" ]; then
 echo "File does not exist, or you do not have read privilge!"
 exit -1
fi

FILESIZE=$(cat "$FULLPATH" | wc -c)
if [ $FILESIZE -ge 300000 ]; then
 echo "The size of file exceeds 300000 bytes!"
 exit -1
fi

FILENAME="`basename "$1"`"
EXT=$(echo $FILENAME | grep -i .gif$ || echo $FILENAME | grep -i .jpg$)
if [ -z "$EXT" ]; then
 FILENAME="$FILENAME.jpg"
fi

URL=$(curl -F "up=@${FULLPATH};filename=c:\\${FILENAME}" \
 -F MAX_FILE_SIZE=300000 -F board=C_Computer -F level=0 -F live=9999 -F "exp=" \
 bbs.njupt.edu.cn/cgi-bin/bbsdoupload \
 | tr -d "\015" | tr -d "\012" \
     | sed "s/^$//g")

echo "\nCopy the following URL to your brower:\n"
echo "http://bbs.njupt.edu.cn/cgi-bin/$URL\n"

Tips in Mex Fortran, linux version

Starting from the day before yesterday, I need to write a Matlab interface for a Fortran source code, which is almost a black box for me, so that I can use "mex" to compile it and call it in Matlab.

The first thing is the compiler. In Windows, you can only choose a specific compiler, say, C/C++ or Fortran. So if you work with both C and Fortran code, you will have to use "mex -setup" from time to time, which is kinda annoying for a lazy person like me. But in Linux, you can work on a configuration file so that Matlab will automatically select the compiler for you.

First, in "/opt/R2009a/unix/bin/mexopts.sh", you can see the following segment:

#
FC='g95'
FFLAGS='-fexceptions'
FFLAGS="$FFLAGS -fPIC -fno-omit-frame-pointer"
FLIBS="$RPATH $MLIBS -lm"
FOPTIMFLAGS='-O'
FDEBUGFLAGS='-g'
#

The first line indicates the compiler for Fortran. "g95" is not provided, so, we will change it to gfortran which I think comes with Ubuntu. Thus, change this segement into:

#
FC='gfortran'
FFLAGS='-fexceptions'
FFLAGS="$FFLAGS -fPIC -fno-omit-frame-pointer"
FLIBS="$RPATH $MLIBS -lm"
FOPTIMFLAGS='-O'
FDEBUGFLAGS='-g'
#

Then, in Matlab, if you type
mex -setup

You will see something like:
The options files available for mex are:

1: /opt/R2009a/unix/bin/gccopts.sh :
Template Options file for building gcc MEX-files

2: /opt/R2009a/unix/bin/mexopts.sh :
Template Options file for building MEX-files via the system ANSI compiler

Choose 2. Then it's ok to compile.

The next thing is the interface. The help in Matlab has everything you know to create a wrapper, even some examples for you to follow. So although I know nothing about Fortran, I managed to write a "almost" complete interface which passed the compilation.

However, there're several important tips that costs me a whole day to figure out.

1. The name of the source file must be ".F", otherwise it won't compile.
2. If the source code you want to wrap was not developed by you, be very careful about all the data types in that code. For example, if one input for that function is "REAL", then it is 4-byte floating number. And in Matlab, unless specified, everything will be "double", which is 8-byte floating number. If you pass the double value directly into it, the value will NOT be preserved. So when you call the function in Matlab, use something like "nlam = int32(nlam);
   
   beta = single(beta);" Also, inside the interface, use something like "mxCopyPtrToReal4(beta_pr,beta,size)".
3. The next thing is managing the output. When you create the output matrices, do not use mxCreateDoubleMatrix. Use "mxCreateNumericMatrix", which will let you to specify the output types like int32 or single.
   

How-To: What should you do if you deleted files in NTFS by mistake

OK. Here is how it happens. One day, I was using the filezilla, pulling down a tiny little file to my hard disk of the desktop machine in my office. It is the major machine I use for research, and all my LaTeX papers, matlab codes are there.

Then, I select the whole "Research" disk, and downloaded that file into it. Then, assuming I am still selecting the tiny file on the remote server, I hit "delete" button. The computer seems to be hanging there with no response, which happens quite a lot when I use filezilla, by the way. So I force quit (thank god).

Then, after maybe a few minutes, I found that all my codes are gone somehow. WTF?!

It was all my fault, can't blame anyone.

So what I did was I delete almost everything in a NTFS disk, when I was using Ubuntu Linux. Bad news. There is no trash can, and everything is somehow "gone forever".

OK, enough for my miserable story.

The first thing I try is under Windows, a commercial software called "Recover my files". It will recover a lot of files, only without the file names. But file names and original locations are SO important because there are thousands of them and most of them are codes, which basically called by its name.

Then, I tried free wares in Ubuntu, such as ntfsundelete and photorec. They all can recover the files, but not the name and location. Well, at least I can recover them, it just takes time. My final choice is photorec.

There is one oerticular software that can recover your file, name and location, though. It is called: "Autopsy". Also a linux free ware, but it works like a webpage, and you can only recover the file one by one. But I did use it to recover a lot of vital source code.

One tip is, after you hurt your disk, you can use "ddrescue" to mirror the whole disk to another externel hard disk (should be much bigger than the one you want to backup). Then, the file you create works just like a disk, and yu can recover based on that without further damage.

The only thing you can do to prevent this from happening is to back up your system regularly. The reason that I can survive this with only one week of frustraion is that I uploaded the most vital part of my work to a remote server, for computation purpose. Because of that, I only need to recover a very small ratio, using the combanation of those softwares.

But starting from now, I will start to use file sync softwares to back up my works to another disk on a regular basis. My recommendation is Grsync. You can setup some commonly profiles, and just click excecute it will sync by only dealing with the differences. Very fast.

You can also write a script to automaticly sync the folders, say, every night. I do not have the time right now. Here are some useful links:

https://help.ubuntu.com/community/BackupYourSystem#grsync
http://www.unixgeeks.org/security/newbie/unix/cron-1.html
http://linuxbasement.com/content/backups-using-rsync-bash-cron
http://linuxgazette.net/104/odonovan.html

Installation of a fancy network visualization tool: Circos

http://mkweb.bcgsc.ca/circos/

I just found this very nice visualization tool, but the installation is kinda tricky :-)

First, you have to have UNIX or Linux on your machine (long live open source!!)

sudo apt-get install libgd-gd2-perl

After downloading and extracting, install these perl modules by:

sudo perl -MCPAN -e 'install Class::Base'
sudo perl -MCPAN -e 'install Math::Random'
sudo perl -MCPAN -e 'install Graphics::ColorObject'
sudo perl -MCPAN -e 'install List::MoreUtils'
sudo perl -MCPAN -e 'install Clone'
sudo perl -MCPAN -e 'install Config::General'
sudo perl -MCPAN -e 'install Readonly'
sudo perl -MCPAN -e 'install GD::Polyline'
sudo perl -MCPAN -e 'install GD'
sudo perl -MCPAN -e 'install Math::Bezier'
sudo perl -MCPAN -e 'install Math::Round'
sudo perl -MCPAN -e 'install Math::VecStat'
sudo perl -MCPAN -e 'install Params::Validate'
sudo perl -MCPAN -e 'install Set::IntSpan'
sudo perl -MCPAN -e 'install Statistics::Descriptive'

Don't copy them to the terminal directly, execute them line by line.

Then, in the cricos-*.** directory, execute
./install-unix
and specify your path of perl (/usr/bin/perl)

Then,
perl ./Build.PL
./Build

In the same directory, try this:
./bin/circos -conf tutorials/2/2/circos.conf

It will create a PNG file in /tmp

I'll start to learn how to use it soon...

Install the GUI in R for Ubuntu

First,

sudo apt-get install r-base

Then, type sudo R in command line to invoke R. Type in

install.packages('Rcmdr')

then

q()

Then use R to invoke it,

library(Rcmdr).

Solving the wired sound problem

My desktop in BioDesign has tons of wired sound problems. I found a fix on the web and actually I don't know if it is a fix or not...

Anyway, I applied it, reboot and MPlayer works again (maybe the reboot works... OTZ...)

sudo apt-get install libsdl1.2debian-pulseaudio padevchooser

(http://ubuntuforums.org/archive/index.php/t-765095.html)

Get Citrix for Linux to work for ASU MyApps

Finally...

First, download and install the Client. It can be found in MyApps.
Then, sudo apt-get install libmotif3

OK. Now we have to import a certificate.
This is the one I use:
http://www.geotrust.com/resources/root_certificates/certificates/Equifax_Secure_Certificate_Authority.cer

http://www.geotrust.com/resources/root-certificates/index.html

and copy it to /usr/lib/ICAClient/keystore/cacerts/citrite.crt
Then it's working :-)

Emacs + LaTeX, even easier

OK.

sudo apt-get install auctex

This thing can give you a whole new style of syntax highlighting dedicated to make your LaTeX life easier.

Changing fonts
C-c C-f C-i \textit{} - Italics
C-c C-f C-b \textbf{} - Bold face (In math mode, this will invoke the \mathbf{} font)
C-c C-f C-e \emph{} - Emphasized
C-c C-f C-s \textsl{} - Slanted
C-c C-f C-r \textrm{} - Roman
C-c C-f C-f \textsf{} - Serif
C-c C-f C-t \texttt{} - Typewriter
C-c C-f C-c \textsc{} - Small Caps
If you want to change font attributes of existing text, mark it as a region, and then invoke the commands.

Sections, Environments, formatting, etc
C-c C-s \section{} \chapter{} \subsection{} - Sections
C-c C-e \begin{} & \end{} - Environment
C-c ] - Close Environment
C-c C-m - Macros
C-c RET(Enter) - Macros
C-c C-j() - Newline with \item inside itemize and so
C-c { - Insert balanced braces
M-q - Fill and indent current paragraph (format it)
C-c C-q C-e - Format current environment

Commenting
C-c ; - Comment current region
C-c : - Uncomment current region
C-c % - Comment current paragraph

Running and viewing
C-c C-c - Run LaTeX on text
C-c C-c after compling - Default View
C-c ` - Go to the next error
C-c C-t C-p - Toggles between DVI and PDF output

Queried search and replace
To do a queried search and replace (with a prompt for replacing each occurrence of a text string), press M-% . You will then be prompted for your search and replace strings. Emacs will find and display each occurrence of the search string and ask you for further instructions. You can respond with any of the following options:
Spacebar Replace text and find the next occurrence
Del Leave text as is and find the next occurrence
. (period) Replace text, then stop looking for occurrences
! (exclamation point) Replace all occurrences without asking
^ (caret) Return the cursor to previously replaced text

If you type any other character, Emacs will stop looking for occurrences and execute the character as a command. To restart the queried search and replace, type:
C-x M- M-RET

How to "Exit" full screen in rdesktop

When you hit ctrl+alt+enter in the RDP session, it should exit the full screen mode... But something in Compiz causes a problem.

System > Prefrences > Advanced Desktop Effects Settings Utility > Workarounds > uncheck Legacy Fullscreen Support

That should do it…

The powerful session manager: Screen

After my supervisor bought this fancy new desktop computer for me, while I am still waiting for the release of Ubuntu 9.04, the first thing I did is installing Microsoft Windows on it. Such a shame thing to do...

Well, I have my reasons though. One thing I assumed that I can "only" do on Windows is Remote Desktop Protocol. VNC is as slow as a snail, ssh -X is really fun except for one thing: it is still a ssh session, which means everything stops if the connection is closed, or, when I just want to shut down my own lap-top. So my requirement is really simple. I want something that is really fast (command line OK), and can still running when I close the current session. Thus, I can setup a program at work, go back home, login to see if everything works fine, modify a little bit if necessary, shut down my lap-top and go to sleep, leaving the server to finish my work.

Is that too much to ask for Linux? No! I realized I just missed one really powerful thing, which has been there (obviously) for a LONG time. It is called "screen". It is just a window manager like xterm, but you can detach from it and reattach anytime you want.

Two really helpful How-Tos:
http://www.rackaid.com/resources/linux-tutorials/general-tutorials/using-screen/
http://www.linuxforums.org/applications/the_screen_program.html

To start a session: type "screen".
inside the session, Ctrl-A ? to see a list of short cuts.
Ctrl-A c for a new window, and Ctrl-A n or Ctrl-A p to switch between opened windows. Ctrl-A K will kill the current window (and the running program).
Ctrl-A d to detach, and then you will find yourself back to the shell, with all your program running in a screen session.

Type screen -ls, you will see all the screen sessions running, and by type:
screen -r SESSION_NAME
You will get back to that session.

Thus, I can just ssh to that server, open a screen with a bunch of Matlab, detach, close that ssh, go back home, re-login and see the output during the process. Yea!

Fixing the Snow plugin in 8.10

The plugins extra nonsupported is the wrong version in intrepid.

Download the latest source from http://releases.compiz-fusion.org/0.7.8/ (plugins unsupported)

Installed dependencies and ./configure --> make -> sudo make install
Also installed the broken plugins extra nonsupported in synaptics (just to replace the broken .so-file later)

Afterwards I replaced the original /usr/lib/compiz/libsnow.so with the new compiled /usr/local/lib/compiz/libsnow.so etc, everything works fine now...

HOW TO: Install Ubuntu on MacBook Pro

Updated to Ubuntu 8.10

First, you should have the Mac OSX in the computer. Coz I have a ipod touch, so I still need the Mac. My successful method for dual boot is:

1. Install the rEFIt Boot Menu from (http://refit.sf.net). After installing it, when you reboot your system, you should see a different grub menu.
2. Use BootCamp to partition the drive (if using Tiger, don't bother burning a Windows boot CD; if using Leopard, it won't ask you). Make some space for the root partition as well as the swap.
3. Restart the computer with Ubuntu Desktop CD inserted (64-bit or 32-bit version). rEFIT should recognize the CD and give you option to "boot Linux from CD". Scroll to this option using the down arrow or click on it with your mouse.
4. Boot Into the live CD, Click on 'Install' icon on the desktop. Select the appropriate language, select 'Keyboard Layout' as 'Macintosh', select 'Manual' partitioning, then 'Forward'...
5. After installation, reboot and you will see rEFIt. But this time the Linux might not be there. Just move the selection and select the rEFIt tools or something, it will ask you if you want to repair the partition table. Repair it, then everything works fine.

In Ubuntu 8.10, the graphic card, the iSight web cam, the wireless network, the bluetooth, and even the multi-touch thouch pad etc. all woks out of the box now, so there will not be any delay for you to enjoy the incredible Intrepid Ibex :-)

For more information:
https://help.ubuntu.com/community/MacBookPro

Skype finally works in 64 bit Ubuntu 8.10

When upgrading from hardy to intrepid, I decided to try out the 64 bit operating system.

Ubuntu really rocks. Even Skype now has the 64 bit version, to install it, first add the Medibuntu to the source list:

sudo wget http://www.medibuntu.org/sources.list.d/intrepid.list -O /etc/apt/sources.list.d/medibuntu.list
sudo apt-get update && sudo apt-get install medibuntu-keyring && sudo apt-get update

then:
sudo apt-get install libdvdcss2 w64codecs skype

Yes, that's right, even w64codecs is available...

But after that, when I tried to open skype to make a test call, it says: problem with audio playback. I restart everything and still nothing. I tried it in the terminal, it says:

ALSA lib ../../../src/pcm/pcm.c:2156snd_pcm_open_conf) Cannot open shared library /usr/lib32/alsa-lib/libasound_module_pcm_bluetooth.so
ALSA lib ../../../src/pcm/pcm.c:2156snd_pcm_open_conf) Cannot open shared library /usr/lib32/alsa-lib/libasound_module_pcm_bluetooth.so
ALSA lib ../../../src/pcm/pcm.c:2156snd_pcm_open_conf) Cannot open shared library /usr/lib32/alsa-lib/libasound_module_pcm_bluetooth.so
ALSA lib ../../../src/pcm/pcm.c:2156snd_pcm_open_conf) Cannot open shared library /usr/lib32/alsa-lib/libasound_module_pcm_bluetooth.so
ALSA lib ../../../src/pcm/pcm.c:2156snd_pcm_open_conf) Cannot open shared library /usr/lib32/alsa-lib/libasound_module_pcm_bluetooth.so
ALSA lib ../../../src/pcm/pcm.c:2156snd_pcm_open_conf) Cannot open shared library /usr/lib32/alsa-lib/libasound_module_pcm_bluetooth.so

I googled a little, and tried to get the 32 bit lib by:

sudo getlibs -p bluez-alsa

This obtained the lib successfully, but I have another error:

ALSA lib pcm_bluetooth.c:1619:(bluetooth_init) BT_GETCAPABILITIES failed : Input/output error(5)
ALSA lib pcm_bluetooth.c:1619:(bluetooth_init) BT_GETCAPABILITIES failed : Input/output error(5)
ALSA lib pcm_bluetooth.c:1619:(bluetooth_init) BT_GETCAPABILITIES failed : Input/output error(5)
ALSA lib pcm_bluetooth.c:1619:(bluetooth_init) BT_GETCAPABILITIES failed : Input/output error(5)
ALSA lib pcm_bluetooth.c:1619:(bluetooth_init) BT_GETCAPABILITIES failed : Input/output error(5)
ALSA lib pcm_bluetooth.c:1619:(bluetooth_init) BT_GETCAPABILITIES failed : Input/output error(5)

Then I got really desperate...

But finally, I changed the options of sound devices, change the Sound In etc from default to HDA Intel, everything works!

Using the malab remotely

It is so disappointing when I found it is so slow to use the ssh -X to run Matlab in my office from my apt. Turns out it is the problem of jre...

Thus, I tried to do it without most of the GUI:
ssh -X
matlab -nodesktop -nosplash

then, it became a command line version of matlab. When I do things like edit x.m, the GUI editor still shows up! And pretty fast.

whos will give all the current variables in the worksapce. So the debugging will be fine.

helpwin give me the GUI help, good.

Matlab Linux can not show command window

After installing Matlab r2007a on my TA computer, one weird problem occurred. It seems that the command window can not be displayed, only a blank window. Anything else seems to be fine, coz when I click start, I can run simulink and so on.

In the command line, the error is:

Locking assertion failure. Backtrace:
#0 /usr/lib/libxcb-xlib.so.0 [0xb558d767]
#1 /usr/lib/libxcb-xlib.so.0(xcb_xlib_unlock+0x31) [0xb558d8b1]
#2 /usr/lib/libX11.so.6(_XReply+0xfd) [0xb591e1bd]
#3 /usr/local/mathworks/sys/java/jre/glnx86/jre1.6.0/lib/i386/xawt/libmawt.so [0xaef5122e]
......

export MATLAB_JAVA=/usr/lib/jvm/java-6-sun-1.6.0.06/jre

then everything goes well again...

Just remember to modify the version of java when updated...

How to get libXm.so.3

apt-get install libmotif3