How to enable hard word wrap in WinEdt 6

WinEdt6

Contrary to the soft wrap, which puts the text into separate lines without inserting line breaks, a hard wrap inserts actual line breaks in the text at wrap points. To enable hard wrap in WinEdt 6 you need to:

  1. from the top menu select Options > Options Interface…;
  2. from the left panel expand “Formatting: Wrapping, Environments…”;
  3. click on “Wrapping.ini”;
  4. locate the SOFT_WRAPPING_FILTER option: SOFT_WRAPPING_FILTER=”TeX;HTML;ANSI;Soft|DTX;INS;STY;TOC;Hard”
  5. remove “TeX;” like follows: SOFT_WRAPPING_FILTER=”HTML;ANSI;Soft|DTX;INS;STY;TOC;Hard”
  6. reload the current script by pressing the icon named “Load Current Script” located in the top-left corner of the options panel on the left;
  7. restart WinEdt.

You can hard wrap the whole document at once by selecting from the top menu:

  • Edit > Format > Format Document

Globus Toolkit 5.0.3 released

Highlights of this release include:

  • GridFTP
    • Added new command: Data Channel Security Context (DCSC)
      • Useful for 3rd party transfers between GridFTP servers that use different CA certificates
    • Added gridftp server chrooting
      • Allows admin to limit the directories a gridftp server can access
    • Added command strings for ‘-disable-command-list’ option for gridftp server configuration
    • Added Progress markers for stream mode
  • Fixed a variety of bugs on GRAM5 related with PBS, Condor and Solaris
  • Updated MyProxy to version v5.3
  • Updated gsissh to version 5.2

[Release notes] – [Software] – [Documentation] -[Support]

Install Globus Toolkit 4.0.8 on Ubuntu 10.04 LTS

Ubuntu 10.04

Disclosure: this tutorial is intended for legacy applications which mandatory need the version 4.0.x of Globus Toolkit. If you don’t have an impelling reason to install that particular version, it is highly recommended that you install the latest version available of Globus Toolkit from its official web site and follow the installation guide of the official documentation.

To install Ubuntu 10.04 server, download its iso from here and create a startup USB drive using UNetbootin (don’t use Startup Disk Creator, its latest version is not compatible with Ubuntu 10.04). Once booted from the USB drive, at the Unetbootin grub menu, scroll down to the same “Install Ubuntu server” option and press TAB. Bring your cursor to the end of this line and press space once, enter

cdrom-detect/try-usb=true

and press enter to start the installation.

  • (1) if you want to build Globus Toolkit download gt4.0.8-all-source-installer from:

http://www.globus.org/toolkit/downloads/4.0.8/

alternatively, if you want to save about one hour, you can use the prebuilt binary installer: you can find it in the right menu of this page. Either way proceed with the next step;

  • (2) run:
root@host:~#
apt-get update

apt-get install vim ant build-essential libpod-* postgresql 
xinetd zlib1g-dev libiodbc2 libiodbc2-dev libxml-parser-perl libxml2-dev
export JAVA_HOME=/usr/lib/jvm/jdk1.6.0_34
export ANT_HOME=/usr/share/ant
export GLOBUS_LOCATION=/usr/local/globus-4.0.8
export GLOBUS_OPTIONS="-Xms512M -Xmx2048M"
source $GLOBUS_LOCATION/etc/globus-user-env.sh
export PATH=$ANT_HOME/bin:$JAVA_HOME/bin:$PATH:$GLOBUS_LOCATION/bin

then log out and back in;

  • (5) run:
root@host:~# service postgresql start
  • (6) open “/etc/postgresql/8.4/main/postgresql.conf”, uncomment the line beginning with “listen_addresses” and set it to listen to all addresses in the following way:
listen_addresses = '*'
  • (7) open “/etc/postgresql/8.4/main/pg_hba.conf” and add the following line at the bottom:
host rftDatabase "globus" "x.x.x.x" 255.255.255.255 md5

where “x.x.x.x” is your public IP address;

  • (8) add a non-privilged user with “globus” as username by runnng:
root@host:~# adduser globus

then add the globus group to all the users you want to use the globus container from:

root@host:~# adduser username globus

then log out and back in;

  • (9) run:
root@host:~# service postgresql restart
  • (10) if you have downloaded the binary installer then extract it into /usr/local/ and run:
root@host:~# chown globus. -R /usr/local/globus-4.0.8/
globus@host:~$ $GLOBUS_LOCATION/sbin/gpt-postinstall

and jump to step (17); if you have downloaded the source installer then proceed with the next step;

  • (11) run:
root@host:~# 
mkdir /usr/local/globus-4.0.8/
chown globus. /usr/local/globus-4.0.8/
  • (12) extract the content of gt4.0.8-all-source-installer you have downloaded into globus’ home directory;
  • (13) log in as globus user or run from terminal “su – globus”;
  • (14) enter the directory where you extracted the content of gt4.0.8-all-source-installer and run:
globus@host:~$ 
./configure --prefix=/usr/local/globus-4.0.8/ --with-iodbc=/usr/lib
make | tee installer.log

this process may take a few hours, depending on your hardware configuration (about one hour on an Intel Core 2 Duo CPU T6500 at 2.10GHz);

  • (15) make a tarfile of /usr/local/globus-4.0.8/ so that you won’t have to rebuild it to install it on other machines; you can untar it on other machines and run $GLOBUS_LOCATION/sbin/gpt-postinstall
  • (16) run:
globus@host:~$ make install
  • (17) get the “distribution package” (a tar.gz file) from your Certification Authority (CA). If you don’t have one you will need to set up a CA in one of your machine following these instructions. Once you have the distribution package copy it on all your server machines and run the following four commands:
root@host:~#
$GLOBUS_LOCATION/sbin/gpt-build your_ca.tar.gz
$GLOBUS_LOCATION/sbin/gpt-postinstall
$GLOBUS_LOCATION/setup/your_ca/setup-gsi
$GLOBUS_LOCATION/bin/grid-default-ca

All the grid nodes on which you want to be able to submit jobs must have installed the public key file (e.g. 0a298c77.0) and the policy file (e.g. 0a298c77.signing_policy) of the CA into the directory /etc/grid-security/certificates;

  • (18) run:
root@host:~#
cd /etc/grid-security/
cp hostcert.pem containercert.pem
cp hostkey.pem containerkey.pem
chown globus. container*
chown root. hostcert.pem hostkey.pem
chmod 644 containercert.pem hostcert.pem
chmod 400 containerkey.pem hostkey.pem
  • (19) still in the /etc/grid-security/ directory create a file named “grid-mapfile” and paste into it the users and hosts of your grid, like in the following example:
"/O=Grid/OU=Monty Python/CN=Graham Chapman" graham
"/O=Grid/OU=Monty Python/CN=John Cleese" john
"/O=Grid/OU=Monty Python/CN=Terry Gilliam" terry
"/O=Grid/OU=Monty Python/CN=Eric Idle" eric
"/O=Grid/OU=Monty Python/CN=Terry Jones" terry
"/O=Grid/OU=Monty Python/CN=Michael Palin" michael

"/O=Grid/CN=host/mercury.domain.com" globus
"/O=Grid/CN=host/venus.domain.com" globus
"/O=Grid/CN=host/earth.domain.com" globus
"/O=Grid/CN=host/mars.domain.com" globus

then run:

root@host:~# 
chown globus. /etc/grid-security/grid-mapfile
chmod 660 /etc/grid-security/grid-mapfile
grid-mapfile-check-consistency
  • (20) make sure your hostname is associated with your public IP. The following entry which is created by default in your /etc/hosts:
127.0.1.1 hostname

should be removed, if your hostname is already mapped to an IP in your DNS server; if it’s not mapped then it should be modified into:

x.x.x.x hostname

where x.x.x.x is your public IP. Furthermore, make sure your parent domain is listed and your DNS servers are properly set in your /etc/resolv.conf

  • (21) create the file /etc/xinetd.d/gridftp and paste into it the following code:
service gsiftp
{
instances       = 100
socket_type     = stream
wait            = no
user            = root
env             += GLOBUS_LOCATION=/usr/local/globus-4.0.8
env             += LD_LIBRARY_PATH=/usr/local/globus-4.0.8/lib
env             += GLOBUS_TCP_PORT_RANGE=50000,55000
server          = /usr/local/globus-4.0.8/sbin/globus-gridftp-server
server_args     = -i
log_on_success  += DURATION
nice            = 10
disable         = no
}
  • (22) run:
root@host:~# service xinetd restart
  • (23) now to configure RFT run:
root@host:~# su postgres -c "createuser -P globus"

enter the globus user password when prompted and then answer in the following way to the following questions:

Shall the new role be a superuser? (y/n) n
Shall the new role be allowed to create databases? (y/n) y
Shall the new role be allowed to create more new roles? (y/n) n
  • (24) run:
globus@host:~$
createdb rftDatabase
psql -d rftDatabase -f $GLOBUS_LOCATION/share/globus_wsrf_rft/rft_schema.sql
  • (25) open $GLOBUS_LOCATION/etc/globus_wsrf_rft/jndi-config.xml (if it’s missing run as user globus the command $GLOBUS_LOCATION/sbin/gpt-postinstall). Look into the “dbConfiguration” section:  make sure the userName value is not “root” but “globus”, then change the content of “password” from “foo” to the password for the globus user you have previously set in postgres. Also ensure that “connectionString” contains the fully qualified domain name of your host (e.g. “jdbc:postgresql://your.host.full.domain.com/rftDatabase”).
  • (26) to check whether RFT is working, start a globus container with:
globus@host:~$ globus-start-container

and leave it running. In a separate terminal run:

user@host:~$ vim /tmp/rft.xfr

Copy and paste the following code, making sure to modify “container.hostname” with your container hostname or IP:

true
16000
16000
false
1
true
1
null
null
false
10
gsiftp://container.hostname:2811/etc/group
gsiftp://container.hostname:2811/tmp/rftTest_Done.tmp

then run:

user@host:~$
grid-proxy-init
rft -h container.hostname -f /tmp/rft.xfr

To check whether the transfer has been correctly performed run:

user@host:~$ diff /etc/group /tmp/rftTest_Done.tmp
  • (27) now to configure GRAM run:
globus@host:~$
vim /usr/local/globus-4.0.8/lib/perl/Globus/GRAM/ExtensionsHandler.pm

and remove the % character in front of $attributes at lines 161 and 174.

  • (28) to give the user globus the privileges to run tasks on behalf of another user run:
root@host:~# visudo

and paste at the bottom the following two lines:

globus  ALL=(username1,username2) NOPASSWD: 
/usr/local/globus-4.0.8/libexec/globus-gridmap-and-execute -g 
/etc/grid-security/grid-mapfile 
/usr/local/globus-4.0.8/libexec/globus-job-manager-script.pl *

globus  ALL=(username1,username2) NOPASSWD: 
/usr/local/globus-4.0.8/libexec/globus-gridmap-and-execute -g 
/etc/grid-security/grid-mapfile 
/usr/local/globus-4.0.8/libexec/globus-gram-local-proxy-tool *

where the user names “username1″ and “username2″ are to be substituted with a comma-separated list of usernames that you actually want the user “globus” to be able to sudo to (that is all the users of your grid). Also, make sure these two entries are each on a single line. If there are any line breaks within an entry then it won’t work;

  • (29) to check whether GRAM is working, start a globus container with:
globus@host:~$ globus-start-container

and leave it running. In a separate terminal run:

user@host:~$ globusrun-ws -submit -c /bin/true
Submitting job...Done.
Job ID: uuid:3304e3f2-55f2-11da-8b8f-00d0b7b7c0bc
Termination time: 11/16/2005 16:09 GMT
Current job state: Active
Current job state: CleanUp
Current job state: Done
Destroying job...Done.

user@host:~$ echo $? 
0

user@host:~$ globusrun-ws -submit -c /bin/false
Submitting job...Done.
Job ID: uuid:456b7c9a-55f2-11da-9b0d-00d0b7b7c0bc
Termination time: 11/16/2005 16:09 GMT
Current job state: Active
Current job state: CleanUp
Current job state: Done
Destroying job...Done.

user@host:~$ echo $? 
1

Globus Toolkit 5.0.2 released

Highlights of this release include:

  • GridFTP
    • Synchronization (globus-url-copy -sync) feature that transfers files only if they do not exist at the destination or differ from the source
    • An offline mode for the server
  • GRAM5
    • Improvements have been made to address all the known blocker issues for production deployment on TeraGrid and OSG
  • MyProxy
    • Updated to MyProxy v5.2

[Release notes] – [Software] – [Documentation] -[Support]

Globus Toolkit 5.0.1 released

Highlights of the 5.0.1 release include:

  • GridFTP
    • New globus-url-sync command for syncing individual files or directories
    • New server option to control the default permissions of created files
    • New server option to time out on slow or hanging filesystems
    • New server logging level to include transfer statistics
  • GRAM5
    • Improved reliability with Condor-G clients
    • Fixed a number of bugs and memory leaks
  • MyProxy
    • Updated to MyProxy v5.1
  • GSI-OpenSSH
    • Updated to GSI-OpenSSH v5.2
  • GSI
    • Added OpenSSL 1.0.0 Support

Also, a Windows installer for GT 5.0 is now available, and includes most standard Globus components like GSI and GridFTP.

Relevant 5.0.1 links:

[Release notes] – [Software] – [Documentation] -[Support]

Globus Toolkit 5.0.0 released

Most components of GT5 are incremental updates (numerous bug fixes and new features) over their GT4 counter-parts (e.g. GridFTP, RLS, MyProxy, GSI-OpenSSH). Some components of GT4 are not included in GT5 (e.g. GT4 Java Core, WS-GRAM4, RFT), to be replaced by new software under development (e.g. Crux, Globus.org Service) […] GT4.x releases will continue to be maintained and supported at least through the end of 2010. New users should immediately adopt 5.0.0. Existing users are encouraged to begin evaluating and migrating to GT5.

via

From a message sent by Ian Foster in October 2009:

  • there have been long struggles and confusion over the GRAM2 vs. GRAM4 components. We have resolved these problems by re-investing in, fixing, and enhancing GRAM2. This new version, called GRAM5, is fully backward compatible with GRAM2 (with two minor exceptions: no support for MPICH-G/MPIG job rendezvous; and GRAM5 stages out stdout/err at the end of the job rather than streaming them out while the job runs), but solves its scalability issues and adds numerous frequently requested features. […] We will continue to support GRAM4 at least through December 2010 (perhaps longer, depending upon demand and funding), but have begun to assist GRAM4 users in migrating to GRAM5. […]
  •  

  • the Reliable File Transfer (RFT) service has been of considerable interest to many Globus users, but in practice has suffered from difficulties in both use and operation. We have decided to replace the RFT functionality with a new Globus.org service: an online hosted service (i.e., Software-as-a-Service), to provide higher-level, end-to-end Grid capabilities, targeted to end users, as well as Grid builders looking for more complete solutions to build upon. Initial functionality of Globus.org will focus on replacing and enhancing the RFT functionality of reliable, high-performance, fire-and-forget data transfer, but over time will grow to include more “collective layer” functionality (as described in the “Anatomy of the Grid” paper 1]). […]
  •  

  • GT4 Java Core is based heavily on obsolete technology (Apache Axis 1.x) and standards (WSRF), yet nonetheless continues to provide tremendous value-add to Web Services-based Grid builders, particularly in the area of security and stateful resource management. With the urging of, and in partnership with, some of our large Java Core users such as the caGrid team at Ohio State University, we have begun the Globus Crux effort to update our Java Web Services stack to newer technologies (e.g., Apache CXF), while preserving and enhancing our core value-add security capabilities as a plug-in to CXF and allowing for WSRF protocol compatibility. […]
  •  

  • While MDS is applicable to a broad range of monitoring and discovery tasks, in practice its predominant use has been to build service registries/catalogs for TeraGrid, caBIG, and BIRN, with limited adoption for systems monitoring in a few other communities. Since MDS4 is intimately intertwined with GT4 Java Core, it would require a substantial reimplementation effort to update it to Crux.  Meanwhile, the state of monitoring tools has evolved considerably since MDS4 was first conceived, with the widespread adoption of highly capable, open source monitoring tools such as Nagios. Therefore we have begun work on a more focused effort to design and implement next generation service registry capabilities using Crux, which we are calling our Integrated Information Services (IIS) effort.  This IIS effort is still in the requirements gathering phase, with no releases planned until sometime in 2010.  We recommend that monitoring needs be met using other tools such as Nagios.
  •  

  • Globus Toolkit version 5.2, targeted for Q1 2010, will focus on repackaging the GT5 components into independent component releases that leverage OS-native packaging approaches (e.g., RPM), with assistance from other groups (e.g., KnowARC) who have already blazed this trail. Subsequent GT releases in the remainder of 2010 will focus primarily on usability and reliability, along with features required by Globus.org. The repackaging effort will not impact backward compatibility with GT 5.0.  We expect GT 5.2 clients and services to be fully compatible with GT 5.0.

Useful Linux howtos

Applications

  • to install Oracle Java JDK follow these instructions;
  • to compare files or directories use “Meld Diff Viewer”. You can install it by running:
    sudo apt-get install meld
  • to install Microsoft’s Core Font package run:
    sudo apt-get install ttf-mscorefonts-installer
  • to synchronize folders, files and make backups use “Conduit Synchronizer”. You can install it by running:
    sudo apt-get install conduit
  • to change grub settings (e.g. to change your default operating system, etc) install “StartUp Manager” by running:
    sudo apt-get install startupmanager
  • to automatically mount at boot your NTFS partitions install “Storage Device Manager” by running:
    sudo apt-get install pysdm
  • to set up a repository cache on your network so that once a package is downloaded from an official repository, all other machines will download it from your local area network, install squid-deb-proxy (and its client so the server downloaded updates get cached by the squid proxy and will also allow the server to install already-fetched updates via the proxy) on your server machine by running:
    sudo apt-get install squid-deb-proxy squid-deb-proxy-client

    then reboot your server machine. Clients can auto-discover caching-enabled machines in the local network by having the squid-deb-proxy-client package installed. To do that simply run:

    sudo apt-get install squid-deb-proxy-client

Information

  • to know your IP address:
    ifconfig | grep Bcast
  • to know your hardware details and save them in an html page run:
    lshw -html > your-file-name.html

    (a good idea is to generate one for each of your computers and save them up for future reference)

  • to check whether a certain package is installed:
    dpkg -l | grep packagename
  • to know whether you need to reboot a machine after updating the system via terminal check if the following file exists:
    /var/run/reboot-required
  • to test the speed of your hard drive run:
    sudo hdparm -t /dev/sd?
  • to list running services run:
    sudo netstat -tulpn
  • to check RAM speed, size and frequency:
    sudo dmidecode --type 17 | more
  • to know your processor details run:
    cat /proc/cpuinfo | grep name
  • to know the bios version of your motherboard run:
    sudo apt-get install smbios-utils
    sudo smbios-sys-info
  • to know the name of the Linux distribution installed on your machine run:
    lsb_release -a

    or, if that doesn’t work then run:

    cat /etc/issue
  • to have some system information of your machine run:
    uname -a
  • to see if your operating system architecture is 32-bit or 64-bit run:
    file /sbin/init
  • to list all users in your system from terminal, run:
    cat /etc/passwd | grep "/home" |cut -d: -f1

Operations

  • to configure your terminal commands history add one or more of the following lines to your ~/.bashrc file:
    • to change name of the file in which command history is saved:
      export HISTFILE=

      (the default value is ~/.bash_history)

    • to not save in the history certain command lines add them as a colon-separated list:
      export HISTIGNORE=
    • to not save a line matching any other previous entry in the history:
      export HISTCONTROL=ignoreboth
    • to change the number of commands to remember in the command history:
      export HISTSIZE=500

      (the default value is 500)

  • to add aliases system-wide you need to edit the file /etc/bash.bashrc For example, the following alias will provide a comprehensive update and clean operation:
    alias apt-update='sudo apt-get -f install
    && sudo apt-get autoremove -y
    && sudo apt-get autoclean -y
    && sudo apt-get clean -y
    && sudo apt-get update 
    && sudo apt-get upgrade -y --fix-missing'
  • to boot in text mode instead of graphical mode (GUI) open the /etc/default/grub file, locate the following line:
    GRUB_CMDLINE_LINUX_DEFAULT=”quiet splash“

    and change it to:

    GRUB_CMDLINE_LINUX_DEFAULT=”quiet splash text“

    and don’t forget to run ‘update-grub’ afterwards to update. If you need the graphical interface you can still have it by simply running “startx” once logged in.

  • to keep your computer’s time accurate follow these instructions;
  • to install fonts for system-wide run:
    mkdir /usr/share/fonts/truetype/myfonts
    cp [fonts] /usr/share/fonts/truetype/myfonts
    fc-cache -f -v
  • to add a new user via terminal run:
    sudo adduser username
  • to add a group to a userrun:
    sudo adduser username groupname
  • to modify the groups of a userrun:
    sudo usermod -G comma,separated,list,of,groupnames username
  • some useful command line shortcuts:
    • Ctrl-D: exit from the current session;
    • Ctrl-R: searching through the command history;
    • Ctrl-A: moves the cursor to the beginning of the command line;
    • Ctrl-E: moves the cursor to the end of the command line;
    • Ctrl-W: deletes the word immediately before the cursor;
    • Ctrl-K: deletes everything immediately after the cursor;
    • Ctrl-Y: undo a deletion.
  • to start a terminal maximized use the command
    gnome-terminal --window --maximize
  • to start a terminal in full screen use the command
    gnome-terminal --full-screen
  • to copy multiple files using scp use it like showed below:
    scp {file1,file2,file3} user@destination:~/
  • to speed up the ssh connection add “UseDNS no” (without quotation marks) at the end of your /etc/ssh/sshd_config and then run:
    sudo /etc/init.d/ssh reload
  • to add a new resolution to your monitor/display/screen run (assuming you want to add 1600×900):
    cvt 1600 900

    that will output something like this:

    Modeline "1600x900_60.00" 118.25 1600 1696 1856 2112 900 903 908 934 -hsync +vsync

    Copy the text following Modeline and paste it in the following command:

    xrandr --newmode "1600x900_60.00" 118.25 1600 1696 1856 2112 900 903 908 934 -hsync +vsync

    Then run the command xrandr alone:

    xrandr

    and note down the connected primary video source (should something like VGA-0, or VGA1). Supposing it’s VGA-0 then run:

    xrandr --addmode VGA-0 "1600x900_60.00"
  • to open/extract an eml file install munpack:
    sudo apt-get install mpack

    then run:

    munpack filename.eml -t
  • to chmod 775 sub-directories, files excluded:
    find /path/to/base/dir -type d -exec chmod 775 {} +

    to chmod 664 files recursively, sub-directories excluded:

    find /path/to/base/dir -type f -exec chmod 664 {} +

Develop Globus grid services with Eclipse

  • Windows – Preferences:
    • Java – Build Path – User Libraries :
      • press the “New…” button;
      • type “globus” as user library name and press OK;
      • press the “Add JARs…” button;
      • select all the JARs in your $GLOBUS_LOCATION/lib directory and press OK.
  • Run – Run Configurations…
    • under Java Application select your application (if you don’t see it in the list then try launching it once first);
    • select the Arguments tab;
      • in the VM arguments text box type:
        -DGLOBUS_LOCATION=${env_var:GLOBUS_LOCATION}

        (This will solve the “Failed to acquire notification consumer home instance from registry” problem)

    • select the Classpath tab;
      • select User Entries;
      • press the Advanced… button;
      • select Add Variable String;
      • press the Variables button;
      • choose env_var;
      • in the Argument field type in GLOBUS_LOCATION
      • press OK and OK.
        (This will solve the “No client transport named ‘https’ found!” problem)

Deployment scripts

In order to be able to develop and test our grid services as easily and quickly as possible, we will need to have:

  1. the full version of Globus Toolkit installed on your development machine;
  2. access to the globus and root users.

[ … under construction … ]

Tips & Tricks

When testing your application with services running on several different machines, it might be handy to:

  • open the terminals inside the same window using tabs (CTRL+SHIFT+T or File/Open Tab);
  • use authentication keys for ssh authentication in order to avoid having to type the same password over and over again. To do so run ‘ssh-keygen’ on your client machine, then copy your ~/.ssh/id_rsa.pub to the ~/.ssh/ directory of every user of every machine you need to frequently log in to;
  • use CTRL+R to search your command history when you need to run a command you have previously run;
  • use CTRL+SHIFT+C and CTRL+SHIFT+V to copy and paste text in terminal;