Locked History Actions

DevNewsBriefs/2015_01_13

All News Briefs
Distribution Summary 2015_01_13

http://getgalaxy.org
January 13, 2015 Galaxy Distribution News Brief



getgalaxy   

getgalaxy.org

galaxy-dist.readthedocs.org

bitbucket.org/galaxy/galaxy-dist

new:

$ hg clone https://bitbucket.org/galaxy/galaxy-dist#stable 

upgrade:

$ hg pull 
$ hg update latest_2015.01.13


Security

Several critical security vulnerabilities were recently discovered by Bartlomiej Balcerek and Mateusz Stahl at the Wroclaw Centre for Networking and Supercomputing. Details regarding these vulnerabilities are provided below, and this stable Galaxy release contains fixes for those vulnerabilities. The Galaxy Team strongly encourages Galaxy server administrators to update their Galaxy servers immediately.

Because of this disclosure, the Galaxy Team performed an extensive audit to identify and fix security issues. Most notably, a large amount of work was done to secure the Galaxy server against cross-site scripting attacks.

Unless otherwise mentioned, the following security fixes have been applied to the current (January 13, 2015) and previous (October 6, 2014) Galaxy releases, identified by the latest_2015.01.13 and latest_2014.10.06 tags respectively.

Arbitrary code execution

A vulnerability was discovered that would allow a malicious person to execute arbitrary code on a Galaxy server. The vulnerability was due to gaps in Galaxy's command line template parameter sanitization. Although all form fields were sanitized for shell metacharacters, some other parameters that might be provided to tools on the command line (such as the input dataset name) were not. Because of this, dataset names and other fields could be constructed to exploit this vulnerability.

Due to the severity of this vulnerability, the fix for it has been applied back to the previous releases beginning with the January 13, 2013 release. The fix can be obtained by executing hg pull && hg update latest_<YYYY>.<MM>.<DD>, replacing the date with the date of the release currently in use.

Cross-site scripting

Many templates used in the Galaxy server did not properly sanitize user input, which would allow for cross-site scripting (XSS) attacks. In this form of attack, a malicious person can create a URL which, when opened by a Galaxy user or administrator, would allow the malicious user to execute arbitrary Javascript and gain access to the user or administrator’s Galaxy account.

OpenID redirect

Additional security has been added to the OpenID authentication methods to prevent a malicious person from redirecting a user to a site other than the Galaxy server from which the request originated. This issue did not cause the exposure of login credentials or provide a malicious person access to a user’s account, but it could be used to trick a user into entering their credentials on a fake Galaxy server.

Mobile Galaxy

Galaxy’s mobile interface, in addition to being vulnerable to XSS attacks, has not been updated with the standard UI, and was largely unusable. Because of this, the mobile interface has been disabled.

Highlights

IPython Integration

Thanks to the awesome work of community members Björn Grüning and Eric Rasche, Galaxy now features integration with the popular IPython project. The Galaxy-IPython project has been merged into Galaxy core and made into a generic plugin framework of interactive environments based on Docker. The IPython plugin allows users to launch and securely connect to an IPython server running in a Docker container, fetch data from their Galaxy history, use the full-feature IPython runtime environment to analyze it, and finally push results back into their history. A YouTube video of the plugin in action can be found here. Information on enabling this plugin is linked to via this Trello card. Interactive Environments (IEs) need to be set up.

Tool Form Upgrade (for Beta Testing)

Galaxy's tool form forces pages to reload entirely in response to many user interactions. This limits Galaxy's responsiveness and can result in a cumbersome user experience when entering complex tool configurations. In Galaxy's development branch, this tool form has been redesigned and modernized to address these and other limitations. This new tool form will become the default with the next release - but we are hoping tool author's and power users enable it and provide feedback during this release cycle in order to ensure it is working ideally when it becomes the default. The tool form can be enabled by setting toolform_upgrade=True in Galaxy's config/galaxy.ini.

New and Improved

  1. New Toolshed category for combinatorial selections tools. https://trello.com/c/QKKYov6a

  2. Updated Admin Tool Panel with options to load and watch directories of Tools and to trigger auto-reload upon update of any tool. https://trello.com/c/fXGdYmzo

  3. Fixed bug in Sentry where it was not generated a URL for history_contents. https://trello.com/c/5d6j4X5y

  4. Added a configurable Google Analytics tracking tag to Galaxy and Tool Shed forms. Requires configuration modifications for full functionality, see ticket. https://trello.com/c/R3agjM9U

  5. Added an API call to collect History Dataset job metrics. Update will let an API user view the id of the job that produced a History’s Dataset. Thanks to Nicola Soranzo. https://trello.com/c/mhmRhxIZ

  6. Updated Job Command line abstraction class to better mask internal values. See ticket for details. https://trello.com/c/IlSdD0qV

  7. Enhanced job scheduling through new process that resubmits jobs terminated due to memory constraints. https://trello.com/c/7dJIHzKd

  8. Enhanced client security. Contains a pair of changes aimed at protected clients by preventing Clickjacking and informing browsers to not send session cookies issues over HTTPS to HTTP URLs by using using SecureFlag. https://trello.com/c/Gt98iH7u

  9. Added RPy requirement to the ngs_simulation tool. Contributed by Björn Grüning. https://trello.com/c/kojo4KR2

  10. Updated to allow Background and Plugin Driven Scheduling of Workflows. API call details included in tickets. https://trello.com/c/wiND0YQh https://trello.com/c/SX2ghmtM

  11. Enhancement to configuration that permits the resolution of relative paths in tool data configuration and .loc files. The string ${__HERE__} will be expanded out to the directory the file (XML configuration or loc) currently resides in. Ticket includes details/use cases/dependencies. https://trello.com/c/5VQOWgld

  12. Improved handling of Report applications. Will discover and utilize proper configuration file. Thank you for testing by Eric Rasche. https://trello.com/c/aRQglAkf https://trello.com/c/SOe8W2U6 https://trello.com/c/SOe8W2U6

  13. Added a javascript validation for username and email changes. A user account was deactivated upon an email address update when no prior activation token existed, requiring account activation, but no notice was given to the user. Now, if no token present (legacy auto-validation), the email must be verified and upon login, the email to do so is sent. Plus minor tunings. https://trello.com/c/WTSZtxuD https://trello.com/c/HJsfz3no

  14. Added blank string for host_url to tooltip rendering when the value is unavailable. Avoids an occasional issue that comes up in the Workflow editor. Thank you Kyle Ellrott. https://trello.com/c/g5xNIYGS

  15. Included enhancements to the "Convert delimiters" and "velvetg" tools. Please see ticket for the changesets included. Thanks to Nicola Soranzo. https://trello.com/c/4cDu8T4I

  16. Expanded configuration options to permit a dynamic external proxy manager (dynamic_proxy_external_proxy), which is required for certain server type’s to have correct functionality/URL paths. Contributed by Eric Rasche. https://trello.com/c/C7wtcdvd

  17. Updated the tool_conf.xml.sample file as the initial phase of Galaxy’s tool revision plans for 2015. https://trello.com/c/soeyuJfV

  18. Implemented a way of creating a Tool Shed repository revisions through an API tarball upload. Overall goal is to simplify tool loads. Please see the ticket for full details. Thank you Björn Grüning. https://trello.com/c/rvO8CijI

  19. Included new API method  /api/tool_data to allow for the interactive interrogation of the tool-data tables on a server. Authored by Kyle Ellrott. https://trello.com/c/zf6Rni64

  20. Improved the docstring of previously added Dataset type detection (sniff) functions. Thanks to Björn Grüning. https://trello.com/c/UFjkigva

  21. Added .xlsx as a new datatype. This enables proper identification and labeling of the (primarily) Excel data upon Upload. Thank you Hunter Moseley. https://trello.com/c/qxGGQ1Ls

  22. Added .cbx as a new datatype. This is used/produced by recent versions of the Cufflinks RNA-Seq analysis tool set. Contributed by Björn Grüning. https://trello.com/c/WBWxACyr

  23. Added .owl and .obo as new datatypes. This facilities data use in the recently created Ontology Toolkit wrappers. Design and testing from Björn Grüning, Erick Antezana, and Peter Cock on behalf of the IUC. https://trello.com/c/4t96N2eV

  24. Updated Docker to run with 'auto-remove' by default (—rm} flag). Containers are automatically removed, which prevents a collection of old work containers from building up. Thanks Kyle Ellrott. https://trello.com/c/uSyg8OYN

  25. Updated Docker to run with 'set user' by default (docker_set_user = true}). This change updates the ownership of commands and any results to be non-root. Thanks again to Kyle Ellrott. https://trello.com/c/0FO0UOe7

  26. Updated Docker to run with the -u $USER argument. Thanks to Björn Grüning. https://trello.com/c/A3VjbvMG

  27. Added tool_library_dir to tool_conf parser (tool_dir was already added). For tool_library_dir, the parser scans the child directories of the given directory, and loads the .xml files inside of them. This permits the loading of all the .xml tool definitions within a the same base directory, included nested directories. https://trello.com/c/OJelgFPu

  28. Allow Model objects to be loaded when they have problematic JSON values. Now, when such a value is encountered, it is substitute with None. https://trello.com/c/9lvIKGXa

  29. Changed the JSON custom type to be a large blob type when MySQL is used. Upgrading migrates the update. https://trello.com/c/RbW6pOd2 https://trello.com/c/qhGD4sIk

  30. Adjusted data column parameters that pointed to »multiple« data parameters. Avoids a server side exception while it builds, validates, and uses a meaningful set of columns. https://trello.com/c/0CCy6mtk

  31. Added a the tool package download function to the API. Update also resolves a few issues in the packaging code. Thank you Kyle Ellrott. https://trello.com/c/7cE1oqmM

  32. Revised SRMA tool wrapper to that it requires at least 2048 MB of memory and reset the tag VALIDATION_STRINGENCY=LENIENT (important for many use cases). Contributed by Lance Parsons. https://trello.com/c/MUb4zETD

  33. Citation URLs open as a _blank new window/tab. Prevents (a browser’s) potentially insecure error messages content from opening in the Galaxy UI middle panel (https vs http). https://trello.com/c/kC3rG30a

  34. Library API improved to return only active libraries (avoiding deleted). https://trello.com/c/PCC2lkHk

  35. Better handling of tool versions updates with significant parameter changes. Regenerate the tool state from parameters on the tool form that are still in common. https://trello.com/c/YfJAzBDI

  36. Reduce minimum length of toolshed repository names from 4 characters to 2. https://trello.com/c/jE7lERZ6

  37. Move handler startup to immediately follow full creation and association of a JobManager. Resolves error where the initialization of the job handler's thread finds that the app has no manager yet. https://trello.com/c/7P5dBqdu

  38. Fixed select2 bug that impacted minimal width. https://trello.com/c/ozKMlL2c

  39. Improved DatasetMatcher to now check if a Dataset’s hda is of the correct format before attempting to perform filtering. This ensures that the correct metadata attributes are intact, with the goal of clarifying job failure reasons (as some attributes may not exist for an unexpected format). https://trello.com/c/wKuW6o1R

  40. Improved handling in the function DynamicOptions AdditionalValueFilter when Dataset columns have not been assigned. Logic now interprets a data’s value instead of failing due to a missing/unassigned name (column label) metadata attribute. https://trello.com/c/kPFaKDlv

  41. Improved handling of the Slurm job CANCELLED state. This improves error reporting, e.g. by clearly stating when a job fails because it exceeds memory quotas versus being cancel by the administrator for other reasons. https://trello.com/c/GA29VWGL

  42. Hide the GALAXY env variable in updateucsc.sh.sample. This enables it to be set externally when calling the script. Useful in docker containers. Contributed by Björn Grüning. https://trello.com/c/rkjT8COY https://trello.com/c/Uu1fDBw2

  43. Relocated job_lock from the queue to the JobManager itself. This fixes NoopQueue from not having job_lock errors when viewed in the UI under Admin → Jobs. https://trello.com/c/iHlVTdMX

  44. Removed r3 instance types due to issues with dependencies that resulted in launch failures. https://trello.com/c/NeqbeLMD

  45. Added flexibility for tool data table configuration in the Install and Test Framework. https://trello.com/c/oKZPySe2

Fixed

  1. Fixed issue where API lost functionality for Twill tool tester that allowed selects to be specified by display value in addition to form value. https://trello.com/c/3opljhof

  2. Fixed issue where logging in after password reset sent the user to wrong page (now point to login). https://trello.com/c/gkZQcy9g

  3. Resolved Admin manage jobs function with accurate time calculation. No longer rolls-over at 24 hrs. https://trello.com/c/7d7e2B1s

  4. Resolved issue where Pages with embedded Visualization were causing a UI error. https://trello.com/c/fZRdzMoI

  5. Corrected issue where installing repositories with many tools causes Galaxy to throw errors in the admin interface. https://trello.com/c/Hv5iIweU

  6. Corrected lib/galaxy/config.py. A missing comma on openid configuration locations has been replaced and the resulting error no longer occurs. Reported by @scholtalbers. https://trello.com/c/AwZwAx4l

  7. Corrected package_picard_1_56_0 so that it no longer contains Picard v. 1.122.0. Discovered by Nicola Soranzo. https://trello.com/c/jKJRjf9N

  8. Finalized a bug fix for over escaping implemented in prior changeset c2bed0a. https://trello.com/c/godTRTgY https://trello.com/c/pAGxM1mb

  9. Fixed a variable name associated with data folders that was causing a NameError issue. Thank you Nicola Soranzo. https://trello.com/c/lXgZR2Yb

  10. Corrected a bug in Internet Explorer (IE) configuration parsing. Thank you Björn Grüning. https://trello.com/c/p49eQLPx

  11. Fixed an improper redirect during user password reset. https://trello.com/c/ROONezok

  12. Fixed Workflow import to correctly set the uuid. Contributed by Kyle Ellrott. https://trello.com/c/4UP6Gfo9

  13. Corrected a few small bugs in docs and pylint. See ticket/changesets for details. Contributed by Nicola Soranzo. https://trello.com/c/HVm9vKl2

  14. Corrected a boolean parameter handling issue that occurred during a Workflow’s runtime execution. ’’Incorporating this fix is critical for proper Workflow execution.’’. The problem manifested as certain tool parameters executing »in the reverse state« when used within Workflows (exclusively, and never when tools were executed directly outside of Workflows). A tool »re-run« form will reveal the issue and various failure errors are known to have resulted. If a prior successful Workflow now fails, and your instance as not yet included in this changeset yet, the issue could likely be the root cause of Workflow tool errors. The problem impacted the Main public Galaxy instance at http://usegalaxy.org for a short time window in November. The fix was applied to the public instance and added to the Stable branch under latest_2014.10.06 upon discovery/resolution, at a priority, during this same time frame. Reported by Andrea Pinna. https://trello.com/c/zdHaxzSn https://trello.com/c/sXUwBJgb

  15. Fixed changeset 04a072e to now use the correct MAKO method in the masthead. https://trello.com/c/ZSMVriGJ

  16. Composite Datatype uploads no longer problematic by assuming groups have a UUID field. https://trello.com/c/GbZwGPmt

  17. Fixed passing nested parameter replacements to the Workflow run API. Discovered by Nicola Soranzo. https://trello.com/c/WtFpviiw

  18. Fixed Pulsar's default HTTP transport to automatically load. Impacts behavior of urllib changes with respect of content length of mmap data after loading. https://trello.com/c/Aq0PK81c

  19. Fixed composite Datatypes issue related to renaming individual parts of paired Datasets. https://trello.com/c/ExOMfxtT

  20. Fixed issue where Dataset download links were being incorrectly populated by regenerating dataset-model URLs upon fetching (even when silent). Plus a corrections for cases where the to_ext value was missing. https://trello.com/c/ngLd7M4u https://trello.com/c/pvdWMBmP https://trello.com/c/pvdWMBmP

  21. Fix Slurm job post-mortem for »clusters« functionality added to slurm-drmaa (and currently in use on http://usegalaxy.org). https://trello.com/c/OuNEdZLc

  22. Fixed datatypes from consuming output extra file paths due to updates in changeset d781366. Resolution aided by James Johnson, Nicola Soranzo & Björn Grüning. https://trello.com/c/gwG6GgW8

  23. Fixed import bug for run_reports.sh. Thank you Nicola Soranzo. https://trello.com/c/ZmTmQTZl

  24. Fixed a parameter parsing issue in the Data Libraries API (recently introduced while refactoring Data Managers). https://trello.com/c/1ZTlAVlT

  25. Corrected a temporary issue where Import was omitted from the original release of latest_2014.10.06. https://trello.com/c/iJwFduar

News and Community

  1. We would like to send a special acknowledgement along with a huge Thank YOU!! (or as our own Dave Clements often states informally, "Hugs!") to our Intergalactic Utilities Commission members. Our project most definitely would not be the same without the IUC's unwavering and dedicated support, contributions, and suggestions throughout the years. Everyone in the Galaxy community benefits directly, in a multitude of ways, that are too far reaching to list out fully in this quick note. Curious about who is involved and the key role this community-driven group has in improving and maintaining the Tool Shed and their owned/reviewed Repositories (in addition to other important areas)? Learn more about the members and future/active/prior projects and goals here....

  2. Explore the latest Galaxy Project news from our team that covers recent Events, Publications, New Tools, and much more in our monthly project reports published in our wiki under Galaxy Updates.

  3. Tool Shed Contributions. This is a brand-new area previously included directly in the Galaxy Updates news letters. Watch as this area develops as we work to summarize new repository updates in a concise and organized format. Feedback about how you would like to see this evolve (including general interest) is welcome. We will be posting a comment/feedback post at Galaxy Biostar to provide an opportunity to for our community to discuss. A summary will be added to Trello once feedback is gathered for review and action. (A link to that post will be updated and added right here in this wiki within the next week - is truly a brand-new endeavor to break this out as a distinct wiki resource!!).

  4. If you are new to Galaxy or wish to connect with our project more in 2015, these key links can help keep you updated about our activities and updates in real time (or at your own pace). Galaxy is a community project we would like to remind all about of the resources and venues available for news and support. Most reading our News Briefs are familiar with Development, Cloud, Local, and other deployment resources such as Admin plus Tool and Tool Repository documenation, but below is a short list of even more places to visit and get connected:

    • All News Reports and the Hub for Distribution Details

    • Twitter (wiki summary) or follow us directly at https://twitter.com/galaxyproject

    • Events, Learn, Galaxy Biostar, Support Resources with FAQ help, Mailing list subscription and archvies, and Vimeo tutorials

    • All know of Public Galaxy Main, but have you reviewed the Other Galaxy Public-hosted Servers lately?

    • Teach resources are an exciting, growing, and key area for expansion throughout 2015, check out what is new!

    • Follow current development real-time and create, comment, and vote on active Trello tickets. As an open source project, we very much welcome community involvement. Not sure how to get involved or how to create an account? We have guidance available here..., that includes a form to aid with quick ticket submission.

    • Community resources. Overview about how we value and seek your input. Have your voice heard and get involved!

    • Galaxy Project home page (hub for all resources, those listed above and more!)

    • See our wiki's right side bar menu → for more links to areas of interest to you

    • Our wiki is absolutely open for community contributions and improvements. We have plans in place for documentation updates in the upcoming year, but we greatly value the knowledge and insight shared through this resource by all that have ideas to make it even better. Let's work together to expand this wiki to meet the needs of the upcoming year as our project matures, as new research/development areas come up, and as Galaxy grows and evolves with new features and enhancements! Create a wiki account and contact us at "outreach at galaxyproject dot org" to become a wiki editor.

Upgrades

  1. Review instructions for core build: Getting the Stable Distribution

  2. Review reset for Toolshed repos: Resetting Metadata For Installed Repositories

About Galaxy

UseGalaxy.org
About GalaxyGalaxy ProjectAdminIssuesBig PictureCommunityGet GalaxyCloudManTool ShedDevelopNews BriefsServersLearnSupportGalaxy BiostarNewsTwitterEventsTeachCiteGalaxy Team