Wurst notes 10 nov 2005 rcsid = $Id: server_notes,v 1.1 2007/09/28 16:57:19 mmundry Exp $ Relevant files -------------- machine file role.. webserver (ness/vesuvio/...) /www/wurst/index.php provide interface very little filtering webserver /var/wurst/submit.pl writes a (/bin/sh) shell script to /tmp Feeds this to batch system. Removes shell script. student machines ~/wurst_server/wurst/scripts/wurst_server.pl Runs the job and mails results back to original user. index.php --------- The original script was built by Axel Griewel. Working on this script has been full of problems with permissions. We are still lacking proper permissions on the directory to put test scripts or for editors to write backup files. For these kinds of reasons, we keep this script very simple and avoid working on it. This script provides the interface and returns an error when something goes wrong. The only jobs it does are * checking for the most obvious errors in the input * check that the email address is valid * executes submit.pl, via a call to ssh. After this script, you lose the connection to the web interface, so initial error checking must be done here. The call to ssh is used to change user id. It is ugly, but our sys admins did not want to use sudo or to set permissions. This has the consequence that anything created by the script has uid "wwwrun". submit.pl --------- This script writes a batch job and feeds it to the batch system. It can only pass success/failure codes back to index.php and it is important that it return an error message if the batch system seems broken. It must write a temporary batch file. It must delete that script at the end. It is best if it redirect standard out and standard error. When feeding a job to the batch system, this script uses options to * specify the batch queue * the shell * merge stdout and stderr * prevent the batch system from producing empty output files by sending stdout to /dev/null All the information necessary to run a calculation should be encapsulated in command line arguments. These have been generated by index.php, so the main work of this script is to loop over the elements of ARGV. submit.pl is written in perl, but, in principle, one could use whatever language one likes as long as it lets you invoke other processes, control return codes and manipulate the file handles for stdout and stderr. The script is invoked with the uid "wurst". wurst_server.pl --------------- This script lives on the execution machine, but we do the editing on the server, taubenstein. The batch system tries to execute the shell script created by submit.pl. You never see this script. If all is working, it executes out of a spool directory owned by the batch system. It makes no sense to write to stdout or stderr unless you take charge of them. The first thing this script does is grab the file handles and redirect them to files with names given by one of the mktemp() style functions. Any program will encounter errors. Even if the code and input are perfect, it is certain that something will be changed on the server and something will fail. The code has to be able to exit and save any diagnostics to a file. Most of the time, jobs run without problems, so any temporary files should be deleted. Useful output comes in two forms from the wurst server: 1. The human-readable output with a ranking of models. 2. The models that are built. Both of these types of files have to be sent back. We use nail/mailx to send results back: The human-readable output is written to a temporary file. This is piped to nail which uses it as the message body. The model files are sent back as attachments. To start building a server ----------------------- You will need a user id for the server to run on the student machines and the same user id on the web server. You will need to build and test each step independently. All the files used for the wurst server should be world readable.