News and Views from the Makers of Request Tracker
This is a reminder that on Friday September 30th, 2011 Best PracticalSolutions will cease support for the 3.6 series of Request Tracker.
We've talked our sales team into including free basic upgrades from RT 3 to RT 4 if you sign up for a new RT 4 support contract. The new RT 4 support contracts are less expensive and come with lots of great new features.
RT 4 adds many options you can put in your etc/RT_SiteConfig.pm totweak the way your RT looks and behaves for your users, as well as how RT works behind the scenes. Some of the new options are also available as individual user preferences by going to the Logged in as … → Settings → Options page.
In this blog post, we'll provide an overview of most of the new options.
These options are discussed in our blog post on autocompletion.
Enable this to redirect to the created ticket display page automatically when using QuickCreate.
These options affect the "More about requestors" box on the ticket display page and will be discussed in a separate upcoming blog post about customizing the displayed requestor information. (If you're eager to customize this, read the sections in etc/RT_Config.pm.)
This determines if the "More about requestors" box on the ticket display page shows information about privileged users. By default it only shows information about unprivileged users.
By default, the simple search in the upper right corner only searches active tickets. If you want to search for a specific status, you can include the status in your search. If you want to search all ticekts regardless of status, disable this option.
When only one ticket is found in search, use this to redirect to the ticket display page automatically. Disabled by default.
Display a simple list of who will receive any kind of mail on the ticket reply page, instead of a detailed breakdown by scrip. Disabled by default.
Remembers the last queue you created a ticket in using the "New ticket in" button and queue list at the top of the page. If you haven't created a ticket in your current session, the Default Queue preference is used. Disabled by default.
When set to 1, the ticket display page will omit the ticket history until a link is clicked to load and display the history. This can be useful if you routinely browse long-running tickets but don't need to always read the history. Disabled by default.
Display the ticket create and update pages using two columns on screens that are wide enough. The new two column pages are much more effective at making more information visible on a single page, especially with many custom fields. Enabled by default.
Enable this option to display the Articles search and selection interface on the Ticket Create page in addition to the Reply/Comment page.
Enable this option to hide the search and include boxes from the Article UI. This assumes you have enabled the Article Hotlist feature on at least one of your Classes, otherwise you will have no access to Articles. This option does not hide the Articles link in the Tools menu.
The available flags under %FullTextSearch are discussed in our blog post on fast full-text searching in RT.
Lifecycles are a powerful way to describe custom workflows for your tickets on a per-queue basis. We'll be covering them in depth in an upcoming blog post. If you're feeling adventurous and would like to dive in early, there is a decent amount of documentation in the Lifecycles section of perldoc etc/RT_Config.pm.
Files in this list will be included in the CSS (squished and unsquished) that RT serves. Recommended if you want local CSS changes that apply to all themes. Alternatively, you can use the advanced section of the Theme editor if you don't need a file on disk.
If set to 0, framekiller javascript will be disabled and the X-Frame-Options: DENY header will be suppressed from all responses. This disables RT's clickjacking protection. It is highly recommended you leave this option set to 1 unless you are embedding RT in an iframe from a different domain.
A handful of options were removed in 4.0, because they were made obsolete by new features or we added a better way to change the behaviour. The most used options that are unsupported in 4.0:
The first three of these options affect what is now controlled by lifecycles. By defining your own lifecycle or modifying the default, you can reapply the functionality. The last option, $SuppressAutoOpenOnUpdate, can be customized by modifying the stock scrip On Correspond Open Tickets that ships with RT.
All the options RT supports are documented in etc/RT_SiteConfig.pm, along with their defaults. To set any of them to a different value for your installation, add the appropriate Set( $OptionName, 'value' ); line to your etc/RT_SiteConfig.pm, clear the mason cache, and restart your webserver. Be sure to only have one Set(...) line for each option, otherwise only the first one will take affect.
I'm happy to announce that RT 4.0.2 is now available.
Continuing with our goal of faster release cycles and smaller changes between releases in a stable series, this release primarily contains fixes for a number of minor bugs. It also includes documentation updates and removal of an inefficient javascript minification option. Notable changes include:
Ability to reference global CFs by Name in RT::Action::CreateTickets
Installation of the docs/ directory into /opt/rt4/docs
Removal of the incomplete --binary flag option for the full-text search indexer
Fixes for a regression that caused group dashboards to vanish after creation and not appear in the list of dashboards
Rewritten forward functionality to generate mail that better represents the original messages received by RT
Removal of the pure Perl Javascript::Minifier module which slowed down
the first request to new webserver children. jsmin or another
external minifier is now required to minify RT's javascript. Refer to
the section about $JSMinPath in perldoc /opt/rt4/etc/RT_Config.pm
for how to configure jsmin.
Best Practical Solutions provides unparalleled instruction in how to get the most out of RT.
2011 will bring two-day training sessions to six cities across the world. As we like to keep class sizes relatively intimate, register soon or we may not be able to guarantee you a seat.
These sessions will be offered in:
If you can't make it to these cities, please drop us a line to request a public training for your area in 2012.
This training will introduce you to the new features in RT 4 as part of a comprehensive overview of RT. Whether you're an old hand at RT or a recent convert, you'll have a good understanding of all of RT's features and functionality by the end of the session.
The first day starts off with a tour of RT's web interface and continues with a detailed exploration and explanation of RT's functionality, workflows and configurability. We'll touch on basic administration, but concentrate largely on helping you and your team get the most out of your RT instance.
The second day of training picks up with RT administration and dives into what you need to safely customize and extend RT. We'll cover point-and-click configuration, upgrading and installing RT, development best practices, RT's API, building an extension, and database tuning.
It goes without saying that you'll get the most out of training if you attend both days of the course, but we've designed the material so that you can step out after the first day with a dramatically improved understanding of how to use RT or show up on the second day and get quickly up to speed on how to make RT do your bidding.
Pricing and Payment
The cost of the class includes training materials, a continental breakfast and an afternoon snack. Please note that lunch will not be provided.
Please contact us at training@bestpractical.com for discounted pricing if you are from an academic institution or if you'd like to send more than 3 people.
If you'd like to pay with Visa, MasterCard or Discover, please visit Best Practical's online store.
If you'd prefer to pay with a purchase order, please email us at training@bestpractical.com.
Be sure to include:
We've heard a few questions from folks who are confused about upgradingto RT 4. You can upgrade to RT 4 and keep all of your data that you created with your current RT. There are no restrictions on bringing that data forward. If you've made local customizations to RT, you'll need to review those and see if we've added your favorite feature. If you're using extensions, you should check to see if those extensions are available for RT 4.
To learn more about what's new in RT 4, consider joining us at one of our training sessions.
While preparing for your upgrade, you should start by reviewing the README file we ship to see if anything has changed since the last time you installed or upgraded RT. You should also review our upgrading docs; there is one such file per major version of RT. You will want to be sure that you read the file for your current version, as well as each following version. If you're upgrading from RT 3.6 to RT 4.0, this means reviewing UPGRADING-3.6 UPGRADING-3.8 and UPGRADING-4.0. These documents are all included in the tarball you downloaded, as well as on github from the links above.
If you're using MySQL and upgrading from RT 3.6 or older, you'll notice that the documentation points you to UPGRADING.mysql. This is a standalone step for MySQL, and you must complete all of the other upgrading steps in addition to the MySQL specific steps. There are no extra steps for PostgreSQL or Oracle which is why we have not included UPGRADING.pg or UPGRADING.oracle files.
We highly recommend that you do a test upgrade. If you're using VMs, you can just snapshot your current RT server and test on that. This will give you a sense of how long your downtime window will be, as well as giving you an opportunity to build a checklist of exact steps for upgrading your specific install of RT. If you're not in a VM, you'll need to find some spare hardware. If you must upgrade on the same hardware, please test your backups.
Now that you have a test system and have extracted the relevant steps
from our documentation, you're ready to start migrating. First review
the output of ./configure --help and see if you need to change any of
the defaults (such as your database name or local users or web
handlers). You'll note that we strongly recommend installing into
/opt/rt4, not installing over the top of /opt/rt3, since we've
removed a number of files and renamed others; things will break if you
install over the top of an existing RT with all of its files intact.
We recommend that you turn off access to RT during your upgrade. This means taking down your webserver. You may also want to configure your mail transport agent to queue mail rather than trying to deliver while RT is down. Make sure you have a final known-good backup and you're ready to begin the final cutover.
Once you've installed into /opt/rt4 and checked
/opt/rt4/etc/RT_SiteConfig.pm to port over your local changes and add
any new features you
want to take advantage of, you can start on your database migration.
You'll use the rt-setup-database command from step 6 of the
README. You
may be stopping halfway through to follow the
UPGRADING.mysql
steps, and then you'll run rt-setup-database again to finish all the
steps. The rt-setup-database command takes care of upgrading core
data inside RT, as well as adding new columns and indexes that RT 4 relies on.
After rt-setup-database, you'll run any of the standalone upgrade
steps (such as the articles importer if you previously used RTFM, the
transactions shrinker or the password upgrader if you haven't installed
the previous security updates).
After that, you're ready to review the web deployment documentation and update your current Apache configuration or switch to one of the other systems we support.
After you've completed your upgrade, you may need to reroute your mail (if you've changed server names or actual servers) and ensure that any cron jobs are running successfully.
If you're having any trouble, please post to the mailing list with details of the errors you're encountering. If you need professional support, don't hesitate to drop us a line at sales@bestpractical.com.
RT is commonly used to manage a lot of data. Unfortunately, prior toRT 4, it didn't support effective indexing of the content of comments or correspondence. RT 4 makes use of the full-text indexing capabilities of PostgreSQL and Oracle, or the external Sphinx library for MySQL, to efficiently index this content, allowing for high-speed searching of textual content. Due to bugs in version 4.0.0, you'll need version 4.0.1 or higher to take advantage of the full-text indexing detailed below.
By default, with no full-text indexes enabled, RT 4 does not allow
searching of the "Content" field at all, owing to how resource-intensive
unindexed full-text searches are. If you wish to enable non-indexed
full-text searching, like previous versions of RT allowed, you will need
to add the following to your RT_SiteConfig.pm:
Set(%FullTextSearch,
Enable => 1,
Indexed => 0,
);
The real magic comes from enabling your database's full-text indexes, however. For the section below, we'll be showing the configuration for PostgreSQL; we'll briefly address MySQL, which is slightly more complicated, at the end of this post.
If you intend to enable full-text indexed searches, you should also read
the docs/full_text_indexing.pod file which shipped with your RT
install, which will provide roughly the same information as below, but
is additionally guaranteed to be up-to-date with your version of RT.
To start off with, you'll need to configure your database to store the
index. On PostgreSQL, this consists of adding a new column and creating
an index on it. To do this, run rt-setup-fulltext-index:
$ /opt/rt4/sbin/rt-setup-fulltext-index
Enter the name of a DB table that will be used to store the Pg tsvector.
You may either use the existing Attachments table, or create a new
table.
[Attachments]:
Enter the name of a column that will be used to store the Pg tsvector:
[ContentIndex]:
Generally, you'll want to store the new column in the Attachments table, right next to the data it is indexing. If you want to back up the index separately, or have other storage constraints, you may wish to create a new table to store it, but the default is almost certainly correct. Similarly, the default column name is also most likely correct.
You may choose between GiST or GIN indexes; the former is several times
slower to search, but takes less space on disk and is faster to update.
[GiST]:
You can read more about the differences between the index types in the PostgreSQL manual.
Going to run the following in the DB:
ALTER TABLE Attachments ADD COLUMN ContentIndex tsvector
Going to run the following in the DB:
CREATE INDEX ContentIndex_idx ON Attachments USING gist(ContentIndex)
You can now configure RT to use the newly-created full-text index by
adding the following to your RT_SiteConfig.pm:
Set( %FullTextSearch,
Enable => 1,
Indexed => 1,
Column => 'ContentIndex',
Table => 'Attachments',
);
At this point, your database has been configured. Ensure that you add
the Set(...) block that rt-setup-fulltext-index output (which may be
different from the above!) to your RT_SiteConfig.pm before continuing.
Your index now exists, but is empty. In order to fill it, we must find
and index every textual attachment in RT using rt-fulltext-indexer
--all. This can be a very time-consuming task, so be prepared for it
to take a while. It can be interrupted and resumed safely, however.
Once it has finished, you will need to schedule rt-fulltext-indexer to
run at regular intervals to pick up new additions to the database. This
is easiest done by running ln -s /opt/rt4/sbin/rt-fulltext-indexer
/etc/cron.hourly, which will ensure that it runs once an hour, but many
other alternatives are possible. Depending on how fast tickets are
created in your system, running it as frequently as once every two to
five minutes may be possible.
The story on MySQL is unfortunately considerably more complicated, as
MySQL does not natively support a full-text index format. It integrates
with the external Sphinx search library, but
this requires recompiling MySQL to include the SphinxSE engine which
allows MySQL queries to retrieve data from the Sphinx searchd indexer.
While many vendors package the external Sphinx tools, none of them ships
a MySQL package which has been compiled with SphinxSE at the time of
writing. As such, this is most likely not for the faint of heart, or
those only just getting their feet wet with system administration.
Despite that, the process of recompiling MySQL with SphinxSE is fairly straightforward. Full instructions are provided in the Sphinx documentation. Sphinx 2.0.1 has been tested to work with both MySQL 5.0 and 5.1; it is not compatible with MySQL 5.5 at this time.
Once a SphinxSE-enabled MySQL has been installed, the configuration
proceeds roughly as for PostgreSQL, above. First, you must run
rt-setup-fulltext-index, which will ensure that your MySQL is
correctly configured with SphinxSE, then prompt you for additional
information:
$ /opt/rt4/sbin/rt-setup-fulltext-index
Enter name of a new MySQL table that will be used to connect to the
Sphinx server:
[AttachmentsIndex]:
Enter URL of the sphinx search server; this should be of the form
sphinx://<server>:<port>/<index name>
[sphinx://localhost:3312/rt]:
In order to communicate with the Sphinx searchd server, we create a
virtual SphinxSE table in the database. This table hence needs to know
where your Sphinx searchd server is running. The default is usually
correct if you are planning to run searchd on your database server on
the default port.
Maximum number of matches to return; this is the maximum number of
attachment records returned by the search, not the maximum number
of tickets. Both your RT_SiteConfig.pm and your sphinx.conf must
agree on this value. Larger values cause your Sphinx server to
consume more memory and CPU time per query.
[10000]:
This is an unfortunate limitation imposed by the design of Sphinx; 10000
should be sufficient for most purposes, but you may need to increment
this number if you have tickets with an extremely large number of
transactions which match your common search criteria. If you wish to
change this number later, you must alter it in both RT_SiteConfig.pm
and sphinx.conf files.
Going to run the following in the DB:
CREATE TABLE AttachmentsIndex (
id INTEGER UNSIGNED NOT NULL,
weight INTEGER NOT NULL,
query VARCHAR(3072) NOT NULL,
INDEX(query)
) ENGINE=SPHINX CONNECTION="sphinx://localhost:3312/rt" CHARACTER SET utf8
You can now configure RT to use the newly-created full-text index by
adding the following to your RT_SiteConfig.pm:
Set( %FullTextSearch,
Enable => 1,
Indexed => 1,
Table => 'AttachmentsIndex',
MaxMatches => '10000',
);
It will also output a sample sphinx.conf file. At this point, your
database has been configured. Ensure that you add the Set(...) block
that rt-setup-fulltext-index output (which may be different from the
above!) to your RT_SiteConfig.pm before continuing.
Your index now exists, but is empty. In order to fill it, we must run the Sphinx indexer:
$ indexer rt
Sphinx 2.0.1-beta (r2792)
Copyright (c) 2001-2011, Andrew Aksyonoff
Copyright (c) 2008-2011, Sphinx Technologies Inc (http://sphinxsearch.com)
using config file '/etc/sphinx.conf'...
indexing index 'rt'...
collected 133809 docs, 1015.6 MB
sorted 137.2 Mhits, 100.0% done
total 133809 docs, 1015645236 bytes
total 144.967 sec, 7006036 bytes/sec, 923.02 docs/sec
total 400 reads, 6.474 sec, 637.6 kb/call avg, 16.1 msec/call avg
total 582 writes, 2.473 sec, 959.3 kb/call avg, 4.2 msec/call avg
Finally, you will need to start the searchd daemon:
$ searchd
Sphinx 2.0.1-beta (r2792)
Copyright (c) 2001-2011, Andrew Aksyonoff
Copyright (c) 2008-2011, Sphinx Technologies Inc (http://sphinxsearch.com)
using config file '/etc/sphinx.conf'...
listening on all interfaces, port=3312
precaching index 'rt'
precached 1 indexes in 0.033 sec
You will need to re-index the data using indexer --rotate rt at
regular intervals. Since Sphinx does not do incremental indexing, but
rather re-indexes all content each time it is run, a "main+delta"
split of the
data may be necessary to reduce index update time sufficiently to allow
close to real-time indexing.
I'm happy to announce that RT 4.0.1 is now available.
This release is a bugfix release. It contains a number of notable fixes identified since the 4.0.0 release:
Fixes for MySQL+Sphinx and native PostgreSQL full text search, and improved documentation.
Javascript to forbid running RT in a frame, to prevent clickjacking attacks.
Better detection and hinting of common web path misconfigurations.
Minified javascript with an external jsmin now works in fastcgi and mod_perl deployments. This requires that mod_perl deployments switch to 'SetHandler modperl'; see docs/web_deployment.pod .
Javascript fixes for IE8 in the admin UI.
Multiple warning fixes during the upgrade process.
Removed previously missed rights related to right delegation.
Best Practical Solutions provides unparalleled instruction in how to get the most out of RT.
2011 will bring two-day training sessions to six cities across the world. As we like to keep class sizes relatively intimate, register soon or we may not be able to guarantee you a seat.
These sessions will be offered in:
If you can't make it to these cities, please drop us a line to request a public training for your area in 2012.
This training will introduce you to the new features in RT4 as part of a comprehensive overview of RT. Whether you're an old hand at RT or a recent convert, you'll have a good understanding of all of RT's features and functionality by the end of the session.
The first day starts off with a tour of RT's web interface and continues with a detailed exploration and explanation of RT's functionality, workflows and configurability. We'll touch on basic administration, but concentrate largely on helping you and your team get the most out of your RT instance.
The second day of training picks up with RT administration and dives into what you need to safely customize and extend RT. We'll cover point-and-click configuration, upgrading and installing RT, development best practices, RT's API, building an extension, and database tuning.
It goes without saying that you'll get the most out of training if you attend both days of the course, but we've designed the material so that you can step out after the first day with a dramatically improved understanding of how to use RT or show up on the second day and get quickly up to speed on how to make RT do your bidding.
Learn more on our website: http://bestpractical.com/services/training.html
On Friday September 30th, 2011 Best Practical Solutions will cease support for the 3.6 series of Request Tracker. 3.6.0 was released on June 15th 2006 and the series has received only critical bug and security fixes since June 2008. We will continue to provide critical bug or security fixes until September 30th, 2011 if any are required.
We're currently developing a schedule for future RT series and will publish an end of life policy and schedule as soon as they are complete. The 3.8 series will continue to receive bugfixes at this time, but our development efforts will be concentrated on polishing RT 4.0 and working on the new features we have planned for 4.2.
As a ticket grows long in the tooth, correspondence which quotes the entire history of the conversation can balloon to become extremely large and unwieldy. It can be taxing just to find the new content of incoming email amidst all its history.
RT 4.0 makes this more manageable by folding away quoted content during ticket display. This lets you focus on what's important: the new text. When you do need to see the quoted text, it's still there, hidden behind a click. Now when you look at a ticket, you won't have to mentally skim over quoted lines or paragraphs. Quote folding makes reading a ticket's correspondence and history far more comfortable.
When you expand those folded-away quotes by clicking "Show folded text", you'll see that RT 4.0 now also colors nested quotes differently, so you can easily distinguish who wrote which text. The nesting level of each quote becomes more apparent when colored. Let RT count the angle brackets for you.
Our quote folding is robust enough to understand various quoting disciplines, whether it be top-posting, interleaved, or bottom-posting. You also don't have to stick with > to indicate quotations; RT recognizes many other kinds of quote markers.
Along with quote folding, we've improved how RT 4 displays HTML content in ticket history. One way that a lot of people quote text is by coloring the original text or the new content in their response. To facilitate such quoting, RT 3.8.2 and later display colors specified in HTML mail. RT 4.0 improves our HTML mail handling further. RT 4 understands more ways that mail client generates text styling, especially Microsoft Outlook. We also permit more kinds of styling, such as font-family and font-size.