Skip to content

Fix support for type 'E' multiple response sets in SPSS SAV files #347

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
slobodan-ilic wants to merge 6 commits into
base: dev
Choose a base branch
from
Open

Fix support for type 'E' multiple response sets in SPSS SAV files #347

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
aee63b2
Fix support for type 'E' multiple response sets in SPSS SAV files
slobodan-ilic Sep 30, 2025
230e1ef
fix: multiple mr_sets
slobodan-ilic Oct 11, 2025
65b0c57
fix: empty subvariables
slobodan-ilic Oct 11, 2025
8a7a645
fix non-utf chars in utf-8 encoding
slobodan-ilic Oct 18, 2025
7acc0df
fix fuzzifier errors
slobodan-ilic Oct 18, 2025
779aac9
Fix encoding errors cross-platform
slobodan-ilic Oct 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 24 additions & 8 deletions src/readstat_convert.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,31 @@ readstat_error_t readstat_convert(char *dst, size_t dst_len, const char *src, si
} else if (converter) {
size_t dst_left = dst_len - 1;
char *dst_end = dst;
size_t status = iconv(converter, (readstat_iconv_inbuf_t)&src, &src_len, &dst_end, &dst_left);
if (status == (size_t)-1) {
if (errno == E2BIG) {
return READSTAT_ERROR_CONVERT_LONG_STRING;
} else if (errno == EILSEQ) {
return READSTAT_ERROR_CONVERT_BAD_STRING;
} else if (errno != EINVAL) { /* EINVAL indicates improper truncation; accept it */
return READSTAT_ERROR_CONVERT;

/* Try conversion, with retry logic for invalid sequences */
while (src_len > 0) {
size_t status = iconv(converter, (readstat_iconv_inbuf_t)&src, &src_len, &dst_end, &dst_left);

if (status == (size_t)-1) {
if (errno == E2BIG) {
return READSTAT_ERROR_CONVERT_LONG_STRING;
} else if (errno == EILSEQ) {
/* Invalid byte sequence - replace with '?' and continue.
* This handles files with encoding errors more gracefully
* than failing completely. */
if (src_len > 0 && dst_left > 0) {
*dst_end++ = '?';
Copy link
Contributor

@evanmiller evanmiller Oct 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer https://en.wikipedia.org/wiki/Specials_(Unicode_block)#Replacement_character

dst_left--;
src++;
src_len--;
continue;
}
break;
} else if (errno != EINVAL) { /* EINVAL indicates improper truncation; accept it */
return READSTAT_ERROR_CONVERT;
}
}
break;
}
dst[dst_len - dst_left - 1] = '\0';
} else if (src_len + 1 > dst_len) {
Expand Down
1 change: 1 addition & 0 deletions src/spss/readstat_sav.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,7 @@ typedef struct sav_ctx_s {
#define SAV_RECORD_SUBTYPE_INTEGER_INFO 3
#define SAV_RECORD_SUBTYPE_FP_INFO 4
#define SAV_RECORD_SUBTYPE_MULTIPLE_RESPONSE_SETS 7
#define SAV_RECORD_SUBTYPE_MULTIPLE_RESPONSE_SETS_V14 19
#define SAV_RECORD_SUBTYPE_PRODUCT_INFO 10
#define SAV_RECORD_SUBTYPE_VAR_DISPLAY 11
#define SAV_RECORD_SUBTYPE_LONG_VAR_NAME 13
Expand Down
Loading
Loading