USE13 - Stata module to open Stata v13 data files in Stata v10 (or later)

Description

use13 facilitates data exchange between users who have not updated to the new version of Stata and other users, who have Stata 13 already. Stata is backwards compatible and all new versions read files produced with any previous version. However older versions of Stata do not understand newer formats. A proper way of sending data to older version is to use Stata's command saveold, but it is only an option for the data producer posessing Stata 13 already.

Here is what the users need to know about the assumptions and limitations of use13:

All legacy types present in Stata 10 (byte, int, long, float, double, and str1...str244) are transferred without any loss of information or precision.
Long strings (strF type) which are longer than 244 characters will be truncated to 244 characters (maximum allowed under Stata 10 file format specification 113). A warning is issued if this happens. Format of truncated string variables is set to %str244.
Long strings (strL type) both ASCII and binary are dropped. use13 notifies the user about the variables that were dropped. There are plans to include them into the resulting file (in truncated form) in a subsequent version.
Timestamps: every data file created with Stata (at least in the version 8 - version 13 range) contains a timestamp inside the file. This timestamp is in general different from the date/time of creation or modification as reported by the operating system. Timestamp is not disturbed by this conversion, which means the converted file will retain the original timestamp. Inside Stata a dataset's timestamp can be seen in the output of the describe command. Note that when an imported file is saved from Stata, it will get a new timestamp reflecting the date/time of that operation. This is Stata's normal behavior.
File type 116 is not supported since they are declared "never released" by StataCorp, although one file of that type was spotted out in the wild. use13 will notify the user if the file is in format 116. There are no plans to support this file format due to lack of the documentation and example files.
It is not clear at the moment whether there are any new variable formats in Stata 13 that might be applicable to legacy data types, but incompatible with earlier versions.
Users of Stata 9 may further use the use10 command to downgrade the dataset to the format supported by Stata 9, but with the loss of the variable formatting expanded in Stata 10.
The program has been tested on more than 100 datasets. Any discovered bugs or updates will be posted here if/when discovered.
The program is tolerant to the bug in Stata, which could produce datasets with incorrect value of the variable labels offset at the file's map table. While Stata 13 has been fixed for this problem (see paragraph #2 in 02jul2013 update to Stata 13 here), datasets affected by it might still be in circulation.

Installation

To install use13 type literally the following in Stata's command prompt:

   findit use13

Then click on the link for -use13- in the Stata's viewer and click the -install- link.

The above installation instructions are sufficient in most cases when Stata is permitted to access the Internet. To transfer files to a computer not connected or not permitted to connect to the Internet, manually copy the files from the distribution package.

To install use10 proceed to use10 page for instructions.

Usage

To import a file type the following command in Stata's command line:

   use13 filename

For example:

   use13 C:\Data13\auto.dta

Filenames with spaces or commas must be enclosed in quotes:

   use13 "C:\My Data 13\auto.dta"

If your data in memory is modified, and you want to proceed anyway, specify option clear:

   use13 "C:\My Data 13\auto.dta", clear

Web data is also accessible with this command:

   use13 "http://www.stata-press.com/data/r13/auto.dta"

Datasets accompanying Stata releases are available on Stata Press website.

To get information about the program, type:

   mata use13_about()

Important

Stata's built-in file manipulation commands are not affected by use13, and you will continue getting error 610 on attempt to use them with Stata 13 files. More details in the next section below.

Clarification on built-in commands

StataCorp does not allow modifications to their Stata's built-in commands by the users. It is technically possible to let the user supply a custom procedure to handle conversion of the unknown types of files, but this conversion process:

often requires numerous parameters, of which the built-in command wants nothing to know;
can degrade performance;
in some cases interactive collaboration with the user is necessary to allow selection of proper parameters;
various other situations.

Hence data conversion is usually a separate step before data processing.

In most cases converting the dataset manually with use13 and storing the converted file to process it later is the preferred mode of operation. However in some systems the files may be generated automatically and for example downloaded from a remote system for processing.

use

Specifically, this means that you should not expect that after installing use13 you will be able to e.g. load the Stata 13 data in e.g. Stata 12 with the same command, such as:

use "file13.dta"

Stata still stop with an error 610 "file not Stata format" and this is a normal and unavoidable behavior. Instead you must write:

use13 "file13.dta"

append

Similarly, but perhaps not as obviously, the other Stata's commands (append, merge, etc) will not know anything about the new format, and installation of use13 will not change it. The code of the append command will not be affected in any way, but instead, as described above, you will need to do a two-step process of converting the file first, then executing the actual command. For example, to append a Stata 13 file:

tempfile tmp
preserve
  use13 "C:\My Data 13\auto.dta", clear
  save "`tmp'"
restore
append using "`tmp'"

We can even envelop this code into a new program of its own:

program define append13

  syntax using filename, [*]

  tempfile tmp
    preserve
      use13 "`0'", clear
      save "`tmp'"
    restore
  append using "`tmp'", `options'
end

This primitive command does not support the file list for appending, which was added in Stata 11. There is no problem to do that too. Append is an .ado file, and viewsource append.ado would reveal it's implementation. Each of the two calls to _append should be prepended with the conversion code, similar to illustrated above. If you care about efficiency, more efficient ways to do it exist, but there is no point to pursue them as use13 is seen only to feel in the temporary capability gap.

It is not recommended to replace any commands supplied by StataCorp with any custom modifications, unless you really, really, really know what you are doing. Hence the above command is named append13 and not append, since then it might affect the performance of other parts of the system, which utilize append.

merge

To merge current dataset with a Stata 13 dataset:

tempfile tmp
preserve
  use13 "C:\My Data 13\auto.dta", clear
  save "`tmp'"
restore
merge make using `"`tmp'"'

You will need to sort data just as you normally merge the datasets that Stata understands natively. Note that use13 preserves not only the order of observations of the original dataset, but also the sort order of the original dataset! You do not need to re-sort the converted file before the merge if it was already sorted.

The only exception to this is when the sort key included a strL variable or a strF variable that can't be accommodated in Stata 12 and earlier (longer than 244 characters). Then sorting key will change. But you would need to re-sort the data anyway. The order of observations in the converted dataset never deviates from the original.

other built-in and user supplied commands

For all other commands the principle is the same, if a command requires a dataset name on disk as a parameter, convert that file from Stata 13 format before running this command. Use tempfiles whenever possible if you are doing it on-the-fly (without storing the converted dataset).

Troubleshooting

Despite every effort was made to make use13 robust and error-free, bugs are still possible, just like in any other program. If you are getting an error while executing the use13 command, consider the following:

Are you using the most recent version of use13?
To check for updates consult the SSC website. If use13 was installed from a beta-location, consider the release version. If you installed a beta version from location other than SSC, the ado update command would not know that a newer version is available at SSC archive, and would not report that an update is available. Check the available version at RePEc. Check the version of the installed use13 with the following commands:
```
which use13
mata use13_about()
```
Are you absolutely sure that file is in Stata 13 format?
Stata 13 files must conform to dta specification 117 provided on Stata's website. In particular Stata 13 files must start with the following content:
```
<stata_dta><header><release>117</release><byteorder>
```
If your file starts with a different content, use13 will not read it. You can use Stata's command -type- to check the content in the header of the file:
```
type "filename.dta"
```
Are you sure the file is not corrupted during the transfer?
Make sure the file is transferred properly: use binary mode for FTP transfers. Zip the file before the transfer (checking of the integrity of the file will be done by the unzip-er automatically).
Which software produced the file?
If the file was produced by software other than Stata, perhaps it could be faulty due to bugs in that software? Early builds of Stata 13 were known to produce files not conforming to 117 specification. Stata itself was not affected by this bug, and it was fixed in subsequent updates from StataCorp. use13 contains a workaround for this known problem and should not be affected as well. Resaving these files in updated Stata 13 helps.
Is this file in dta 116 specification?
Files in format specification 116 exist, but the description of it was never released by StataCorp. The converter use13 should produce an informative error message in such a case. This is by desing.
Has Stata's format changed again?
Perhaps the developers of StataCorp have changed the file format again after use13 was released while I was not paying attention? Let me know.
Undocumented features?
There are currently no reasons to believe that there are any features in the _dta file format that are left undocumented.

Necessary information I request:

The following information is usually necessary during the troubleshooting. I would usually point you to this page if any of the items below are not clear.

what was the command resulting in an error, exactly as you typed it?
what is the error message, exactly as Stata reported it?
what is the version/flavor/platform of Stata that you are using?
what is the history of the data file? what software created the file? (name, version, platform, date)
was the file transferred from one computer/system/platform to another?
can Stata 13 itself open the file? (if Stata 13 is available).
do you have only one such file (random error)? or you have a number of these files (error is systematic and likely reproducible)?
if you know what is in the file (how many variables, how many obervations, how long are the long variables if present) please let me know this information.
what is the file size in bytes?
can you make the file available to me? (do not send the data in the first communication)

Further information may be required depending on the type of the problem.

sradyakin/at/worldbank.org

Author

use13 was written by Sergiy Radyakin.

To contact the author send email to: sradyakin/at/worldbank.org.

In case you are experiencing a problem converting a dataset, refer to the following troubleshooting reference first, before requesting assistance.

I am grateful to

for recommending use13 to their users.

Chuck Huber (StataCorp) also recommended use13 to directly read the data from Stata 13 files in the Stata Blog.