What Is eQUEUE?
eQUEUE is a unique software solution that allows system administrators to create easy-to-use web-based job submission forms.
It is designed to increase cluster utilization by bringing more users to the cluster who would ordinarily stay away due to the complexity of submitting jobs to a cluster.
There is no need to learn Linux or scripting. The end user simply inputs his or her data into predefined fields and the job is now in the cluster’s queue to run.
Read: Scientific Computing World article features eQUEUE
Our web-based job submission tool is featured among other services and tools designed to help users manage HPC resources effectively.
“The beauty of this system is that a job submitted through the web interface is just a job like anybody else’s.” – Kyle Sheumaker, CTO and President, Advanced Clustering Technologies
Remove Visualization
Run interactive GUI applications on your cluster nodes using only a web browser.
Analytics / Reporting
Provide useful insights into how your cluster is performing and being utilized.
Access Files
Users provided a web interface to view, download, and tail files. No complicated file copy protocols.
Submitting Jobs
Customizable forms for easy job submission to your high performance computing cluster.
Job Management
Easily see running and queued jobs on the cluster, with simple tools to cancel jobs.
Job Insights
Get valuable statistics on jobs run on your cluster, including cpu, memory, and swap usage.
Job History
Users can easily see all of their previously run jobs. Including statistics on the how it ran.
Detailed Specifications
- Are users staying away because they find it too difficult to submit jobs?
- eQUEUE makes it easy to run jobs on your cluster. Users no longer need to understand complex job schedulers, the Linux command line, shell scripting, etc.
- Administrators create forms using a simple INI format for applications on the cluster (see sample below).
- Users pick the application they wish to run, answer a few questions, and the job is now submitted to the cluster. All with only their web browser.
- eQUEUE utilizes your existing scheduler, nothing new to setup or configure (supports both Torque and GridEngine / Open Grid Scheduler)
- All existing scheduling rules and permissions will still apply. Jobs submitted through eQUEUE act no differently than jobs submitted on the command line.
- eQUEUE’s easy to use forms virtually eliminate any user error when submitting jobs. Creating happy users and a better utilized system.
- eQUEUE isn’t limited to just submitting batch jobs. Interactive GUI apps are also supported.
- Now you can expand the use of your cluster to support interactive applications, including popular applications like: Matlab, VMD, ParaView, etc.
- eQUEUE offers full OpenGL support, perfect for visualizing large datasets already on the cluster. No need for users to download huge files, and install software on their own computers.
How it works
- All interactive apps still run through your scheduler, and follow any permissions or restrictions put in place via those tools.
- No special software is needed on users’s computer, a web browser and the Java runtime with WebStart.
- Interactive sessions are run through an accelerated VNC server, and tunneled via SSH for optimal security. No new ports to open, or unencrypted data.
- OpenGL applications are rendered on compute nodes with 3D capable video cards. Advanced Clustering’s Pinnacle Flex systems are perfect for this.
- Multiple users can now share 3D cards, reducing capital expenditures on expensive workstations. Users now just need any computer (Windows, Macintosh, or Linux) with a modern web browser to access high-end 3D applications.
- Adjustable quality and image compression sliders are available to make interactive applications work even over slow network connections.
Interactive application forms
- Simple forms, just like the ones for batch jobs, are created to start applications, or even entire desktops (GNOME, KDE, XFCE).
- Administrators have full ability to customize these forms with multiple widgets to meet the site or applications requirements.
- Below is a sample form for the VMD molecular dynamics program.
Associations
- Not only do users have the ability to start interactive applications from submit forms, they can do it within eQEUEUE’s file section.
- Administrators can define filetype to application associations via simple regular expression.
- The image below shows how .xyz data files are associated with VMD, and users can easily start a session to view the data.
Viewing the application- Below is a screenshot of the VMD application running via eQUEUE.
- Again because we are taking advantage of our cluster nodes, the user doesn’t have to install any software, or have high-end 3D cards in their own computer. All rendering is being done on the cluster.
eQUEUE provides detailed reports based on an accounting log analysis of your cluster
Insights include:
- Analysis at the user, group, and queue level
- Including, for example, memory utilization across jobs, across users, across groups
- Usage details based on time of day, day of week, and months of year
- Find out when your cluster’s peak usage is, as well as when it’s less active
- Segment your users into custom sets of categories like department, user type, etc.
- Find out who your heaviest users are
- Who tried to submit jobs but were unsuccessful
- Export reports as PDF or CSV file for further analysis or distribution
- eQUEUE gives users an easy way to see the status of jobs running on the cluster.
- No need to use SSH and run qstat, now you can easily see job status via a web-browser.
- eQUEUE gives you full job details, statistics, and the ability to kill running jobs.
- Job management is also integrated with our Job Insight functionality. This gives users real-time information about how their job is performing.
- Users can easily see how much CPU, memory, and swap usage is currently being used for their job.
- Data is presented in easy to understand graph form, so they can see how their job utilizes resources over the lifetime of their jobs.
- Statistics are shown for all nodes in a parallel job as an aggregate, or each individual node.
- Using your scheduler’s accounting data eQUEUE provides users an easy way to see details about their past job.
- Never before has historical job information been so easily accessible to end users.
- Full date range, and text search capabilities make it easy to find details about jobs that were run, yesterday or even months ago.
Job History Statistics
- Easily see how much CPU, memory, and swap was used by past jobs.
- Data is presented in easy to view graph form, making it easy to see how resources were used over the lifetime of the job.
- Statistics are shown for all nodes in a parallel job as an aggregate, or each individual node.
- Never again will a user be able to say they don’t know how much memory, or how long their jobs take to run.
While a job is running, or after it has finished, eQUEUE provides unique, easy-to-use insights into the performance of your jobs.
- eQUEUE gives you a snapshot of your job’s use of CPUs, memory, and swap usage
- You can view the job’s overall performance or by individual node
- Shown in graphical form, you can easily see how your job is performing
eQUEUE also provides a quick glance to show the status of all the queues in your cluster:
- eQUEUE provides access to users’s home directories where you’re able to view and download files directly.
- No need for users to install or learn complicated FTP or SFTP clients. They can now access their files in any web browser.
- The builtin viewing, and tail features provide a great way for users to monitor the output of their job data files, without having to login to the cluster.
- Tail: view your file and automatically update as it changes.
- Download: download the output of the file to your system.
- Additional custom entries: administrators can define filetype to application associations via simple regular expression.