Two new extension commands have been posted to the SPSS Community website (www.ibm.com/developerworks/spssdevcentral) in the Extension Commands collection on the Downloads for SPSS Statistics page.

STATS DBSCAN does density-based clustering with noise handling.  STATS DBPRED uses the results from STATS DBSCAN to predict the clusters for new data.  Density-based clustering depends on a maximum distance parameter and a minimum cluster size in addition to the variables.  It bears a relationship to hierarchical clustering with certain parameter settings, but it is reported to be much faster.

These procedures require the R Essentials.

Regards,

Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
peck@us.ibm.com
phone: 720-342-5621

A new extension command, STATS CANCORR, has been added to the SPSS Community website (www.ibm.com/developerworks/spssdevcentral) in the Extension Commands collection.  It computes canonical correlations and scores, and it can generate an SPSS Statistics syntax command file to score new data.

This command's functionality is similar to the old CANCORR macro, but it includes a dialog box interface, standard SPSS command syntax, and has pivot table output.  It requires the Python Essentials.

Regards,

Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
peck@us.ibm.com
phone: 720-342-5621

new extension command, STATS SURVREG has been posted to the Extension Commands collection of the SPSS Community website (www.ibm.com/developerworks/spssdevcentral).  Click on Downloads for SPSS Statistics and then Extension Commands to access it.  The procedure estimates survival regression for left, right, interval, and counting censoring for these distributions: Weibull, exponential, Gaussian, logistic, lognormal, loglogistic, and t.  It can do Tobit regression as well, although there is another extension command, SPSSINC TOBIT REGR, dedicated to that model.

It can save a dataset with a variety of residuals, predicted values, and diagnostic measures.

The package requires the R Essentials.  It has been tested mainly with Statistics 21 but is expected to work with versions back to 18.

We hope that you will find this package useful.

Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
peck@us.ibm.com
phone: 720-342-5621

A new extension command, STATS_EXRASCH, has been posted to the SPSS Community site (www.ibm.com/developerworks/spssdevcentral) in the Extension Commands Collection.  It estimates standard dichotomous Rasch, linear logistic test, rating scale, linear rating scale, partial credit, and linear partial credit models.  The output includes extensive tables, plots, and test statistics.  It requires the R Essentials, which are also available from the Community site.  The procedure has been tested mainly with Statistics 21, but it is expected to work with versions back to 18.

I would like to thank Ryan Black for his generous advice and review of the preliminary version of this procedure, which has resulted in a substantial increase in its quality.  He is, of course, not responsible for any defects that might remain.

Note:  Since the recent IBM developerWorks upgrade we have received some reports of malfunctioning links.  The dW team has traced this to a bug in Internet Explorer 8 which results in misinterpretation of some URLs used on the site.  If you encounter this problem, switching to Firefox or another browser should solve it.  The team is working on a solution.


Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
peck@us.ibm.com
phone: 720-342-5621

Three Statistics extension commands have been updated.

SPSSINC RANFOR and SPSSINC RANPRED, which produce and predict using random forests, have been rewritten so that they no longer require the Python Essentials.  They now require only the R Essentials.  They are repackaged to use the standard extension bundle installer on the Utilities menu, and they have some new features and improved output.

SPSSINC APRIORI, which builds association rules, has similarly been rewritten to require only the R Essentials and to use the extension bundle installer .  It also has a few new features.

Regards,

Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
peck@us.ibm.com
phone: 720-342-562

As has already been discovered, fixpack1 for Statistics V21 is now available: it was posted sometime Wednesday.  While we like to keep the list informed, this is an informal mechanism and shouldn't be relied on as the official notification mechanism.

I would like to remind users that we welcome contributed code on the SPSS Community site.  Anyone can post something to their (free) account on IBM developerworks - www.ibm.com/developerworks/mydeveloperworks - subject to the reasonable Terms & Conditions posted on the site (no copyright or patent infringement, nothing malicious etc).  If you post a file and would like it listed in one or more of the SPSS Community collections, that is easy to  arrange.  Instructions are on the Welcome page of the site.

If you set up an account, you can also create your own blog on dW.  The SPSS Community blog is not as active as it should be, but the bookmarks on the site list a number of other blog sites with good content, such as Andy W's.  If you know of other good blogs that should be listed, you can send me a note.

Regards,

Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
peck@us.ibm.com
phone: 720-342-5621

A pair of new extension commands for Statistics have been posted to the Extension Commands collection on the SPSS Community website (www.ibm.com/developerworks/spssdevcentral)

STATS GBM estimates generalized boosted regression models including regular regression, logistic, multinomial, poisson, and quantile regression among others.
STATS GBMPRED uses the estimates from STATS GBM to calculate predicted values for new data.

These procedures use the R gbm package and require the R Essentials.  They were developed using Statistics V21 but should work with version 18 or newer.

I am calling these beta versions as there are many features that are difficult to test comprehensively in combination.  Please report any bugs or comments to me.


Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
peck@us.ibm.com
phone: 720-342-5621

The STATS LATENT CLASS extension command has been updated to allow you to save a dataset of scores, i.e., the predicted class, for each input case.  The dataset contains an ID, the maximum probability class, and the probabilities for all the classes.  The procedure can be found in the Extension Commands collection on the SPSS Community website (www.ibm.com/developerworks/spssdevcentral).

Regards,


Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
peck@us.ibm.com
new phone: 720-342-5621

A new extension command, STATS ADJUST WIDTHS, has been added to the Extension Commands collection on the SPSS Community website (www.ibm.com/developerworks/spssdevcentral) under Downloads for SPSS Statistics.  This command synchronizes the widths of selected string variables across different sav files or datasets in order to facilitate file merging and similar activities.  It provides several choices on how to set the widths and various options about what to do with the modified files.  The command requires the Python Essentials and should work with V18 and above, although it has been tested mainly on V21.

Also, I forget to mention in the last news note that the extension.py file installed with the Python Extensions has been updated to address a problem that appears sporadically with some combination of Windows 8 and 64-bit Statistics.  If you have not seen a mysterious error emanating from this module you don't need this update.  If you have, you can get the update from the Python modules collection on the Community site.

Regards,

Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
peck@us.ibm.com
new phone: 720-342-5621

A new extension command, STATS LOESS, has been added to the Extension Commands collection on the SPSS Community site (www.ibm.com/developerworks/spssdevcentral).  It fits a loess regression curve for a model with 1-4 independent variables and saves the predicted values for the estimation data or new data values to a new Statistics dataset.

This command requires the R Essentials and at least Statistics version 18.

Regards,


Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
peck@us.ibm.com
new phone: 720-342-56

A new extension command, STATS CARTPROD, has been uploaded to the SPSS Community website (www.ibm.com/developerworks/spssdevcentral) in the Extension Commands collection.  It computes the Cartesian product of two sets of variables in the same or separate datasets and writes the result to a sav file and opens it.  While not as fast as this operation would be in a database, it is quite general and preserves all variable properties.  (All the data operations are carried out with traditional Statistics syntax, but Python code is used to generate that syntax.)

This command requires the Python Essentials.  The Essentials can be obtained via the Community site or the Statistics product materials, depending on the Statistics version.  The extension command can be downloaded from the Extension Commands collection or installed directly using Utilities > Download and Install Extension Bundles with Statistics version 22.

Regards,


Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
peck@us.ibm.com
phone: 720-342-5621

One new item and one updated item have been posted to the SPSS Community website (www.ibm.com/developerworks/spssdevcentral).

The new item: Apropos of the recent discussions on this list about propensity score matching, a new custom dialog box has been uploaded to the Statistical Tools Collection on the site to simplify doing PSM.  It does a logistic regression following by matching and produces a new dataset of cases and their matched controls.

PSM requires at least version 1.3.0 of the FUZZY extension command.  That version of FUZZY is included in the Python Essentials for Statistics 21 and can also be downloaded from the Community site in the Extension Commands Collection.  PSM has been tested only with Statistics version 21 but may also work with versions 18-20.

The updated item: The version of the FUZZY extension command that includes a single-dataset dialog box interface in Statistics 21 has a bug in the dialog box that disabled fuzz factors.  That has been fixed in the version now posted in the Extension Commands Collection on the site.  The bug affected only the dialog box.

Regards,

Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
peck@us.ibm.com
new phone: 720-342-5621

The SPSSINC TRANS extension command, which essentially allows Python functions to be used as transformations, has been updated.  It now provides control over user missing values, and there is a new subcommand that allows the TO construct to be used in variable lists in the FORMULA subcommand.  (TO could already be used in the results list.)

The new version can be downloaded from the SPSS Community site (www.ibm.com/developerworks/spssdevcentral) via the link to Downloads for SPSS Statistics and then to the extension commands collection.

SPSSINC TRANS is now included in the Python Essentials, but the posted version is newer.


Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
peck@us.ibm.com
new phone: 720-342-5621

In honor of Shakespeare's birthday :-) two new extension commands have been uploaded to the SPSS Community website.

STATS FIRTHLOG estimates the Firth variation of binary logistic regression, which can address the separation issues that sometimes arise in the standard logistic model, especially when the sample size is small.

From the algorithm description...
In general, maximum likelihood estimates are often prone to small sample bias. To reduce this bias, Firth suggested to maximize the penalized log likelihood log L(beta)* = log L(beta) + 1/2 log|I(beta)|, where I(beta) is the Fisher information matrix, i. e. minus the second derivative of the log likelihood. Applying this idea to logistic regression, the score function U(beta) is replaced by the modified score function U(beta)* = U(beta) + a, where a has rth entry a_r = 0.5tr{I(beta)^{-1} [d I(beta)/d beta_r]}, r = 1,...,k.

STATS INEQUALITY calculates up to seven measures of distribution inequality for a variable, including Gini, Atkinson, Theil, and others.  It supports split files, which makes group comparisons easy.

These commands require the R Essentials and at least version 18 of Statistics.  They can be downloaded from within Statistics using the Utilities menu in version 22. For older versions you can download them from the SPSS Community website (www.ibm.com/developerworks/spssdevcentral) in the Extension Commands collection.



Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
peck@us.ibm.com
phone: 720-342-5621

A new graphics extension command, STATS BAGPLOT, has been uploaded to the SPSS Community website.

This procedure produces a bagplot or, if there are more than two variables, a matrix of bagplots.  Bagplots are two-dimensional versions of the boxplot.  Instead of a box, the bagplot displays a convex polygon that contains half of the points (the bag) and a larger enclosing polygon, the loop, whose size is determined by a "fence" parameter.  Points outside the loop are considered to be outliers.

Split files mode is supported for this procedure.

This command requires the R Essentials and at least version 18 of Statistics. It can be downloaded from within Statistics using the Utilities menu in version 22. For older versions you can download it from the SPSS Community website (www.ibm.com/developerworks/spssdevcentral) in the Extension Commands collection.


Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
peck@us.ibm.com
phone: 720-342-5621

I'm trying to install the numpy Python library from the executable files
(http://docs.scipy.org/doc/numpy/user/install.html). When going through the
user prompts, the file chooses to install to a separate location of python
on my machine, and does not allow me to choose the SPSS library location.
(See image at end.)

This is more of a python problem but I figured I would give the folks here a
shot if they knew the answer right away. Is there maybe a way to execute the
.exe file from the command prompt with the correct library location?

I have easy install currently set up to install at the SPSS location, but if
that fails I have a difficult time installing libraries - especially if they
have code that requires compiling different languages (e.g. C++). So I have
successfully used easy install for a few libraries - but no go with numpy.
I've tried setting every imaginable set of environment variables to the SPSS
python library (PYTHONHOME, PYTHON_HOME, PYTHONPATH, etc.) but the SPSS
library location still never shows up as an option in the executable.

(What happened here is that I installed ESRI's ArcGIS - which uses python in
a pretty similar way - after SPSS, so that newer python path wins
apparently). I'm hoping I don't have to muck around in the registry
(https://docs.python.org/2/using/windows.html) - I don't think the IT guy
will take too kindly to that (and I couldn't even find the keys mentioned in
the docs anyway!)

<http://spssx-discussion.1045642.n5.nabble.com/file/n5725718/ScreenInstall.png>

Updated

A new extension command for SPSS Statistics, STATS SVM, has been uploaded to the SPSS Community website.

This procedure creates classification or regression support vector machines using any of four kernel types (linear, polynomial, radial basis, or sigmoid).  The result can be saved and used for predictions.  A grid search over the parameter space can be carried out to find the best parameter set.

This command requires the R Essentials and at least version 18 of Statistics. It can be downloaded from within Statistics using the Utilities menu in version 22. For older versions you can download it from the SPSS Community website (www.ibm.com/developerworks/spssdevcentral) in the Extension Commands collection.



Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
peck@us.ibm.com
phone: 720-342-5621