More on msys-perl fork bug, some positive results

classic Classic list List threaded Threaded
46 messages Options
123
Reply | Threaded
Open this post in threaded view
|

More on msys-perl fork bug, some positive results

lrn-2
Attached a testscript that exposes the
"backticks-with-modified-env-will-trigger-a-fork-failure" bug.
Funny thing about it is that when /bin/perl is used, fork works
correctly. This bug does not seem to depend on perl version - i've built
5.6.1-2 from msys tarball, and it demonstrates this bug as well.
To fix that i've been running tests with /bin/perl AFTER installing it.
That required some fixes to the tests, since some of them do not respect
$ENV{PERL} and force ./perl
I think it is a reasonable assumption that msys-perl will always be
running from /bin

As a result i've got:
Failed Test                     Stat Wstat Total Fail  Failed  List of
Failed
-------------------------------------------------------------------------------
../ext/Cwd/t/cwd.t                 2   512    29    2   6.90%  22-23
../ext/Cwd/t/taint.t               8  2048    17    8  47.06%  2 4 6 8
10 12 14
                                                                16
../ext/Socket/t/socketpair.t      23  5888    45   23  51.11%  22-35 37-45
../lib/File/Copy.t                 4  1024    60    4   6.67%  28-29 55-56
../lib/File/Find/t/find.t                    199   62  31.16%  26-28
38-40 51-
                                                                53 62-64
80-82
                                                                109-113
170 184
                                                                186 188
200-227
../lib/File/Find/t/taint.t         5  1280    45    8  17.78%  26-28 32
46-48
../lib/File/Spec/t/Spec.t                    472    1   0.21%  468
../lib/Net/Ping/t/450_service.t               26    2   7.69%  9 18
../lib/Test/t/multiline.t                      2    1  50.00%  2
lib/filefind-taint.t             255 65280    45   85 188.89%  3-45
op/stat.t                                     86    2   2.33%  9 32
op/taint.t                         9  2304   238  420 176.47%  1-3 5 31-238
64 tests and 286 subtests skipped.
Failed 12/989 test scripts, 98.79% okay. 306/116626 subtests failed,
99.74% okay.

All these failures are either mysterious, or related to symlinks or are
TODOS (multiline).
One is a rebase failure. Not sure why it happens - i've rebased the
contents of perl source tree prior to installing it (rebased with the
rebase address from perlrebase script).


Details:

op/stat.....................................# Failed at op/stat.t line 113
# it should not be '1299234589'
# but it is.
# Check if you are on a tmpfs of some sort.  Building in /tmp sometimes
# has this problem.  Building on the ClearCase VOBS filesystem may also
# cause this failure.
#
# Darwin's UFS doesn't have a ctime concept, and thus is expected to fail
# this test.
# Failed at op/stat.t line 204
FAILED tests 9, 32
     Failed 2/86 tests, 97.67% okay

op/taint....................................F:\Msys-got\1.0\bin\perl.exe: ***
unable to remap
f:\src\perl-5.8.8-trunkfix\perl-5.8.8\lib\auto\Fcntl\Fcntl.dll to same
address as parent -- 0x3F0000
       0 [main] perl 10684 sync_with_child: child 10664(0x108) died
before initialization with status code 0x1
   18818 [main] perl 10684 sync_with_child: *** child state child
loading dlls
F:\Msys-got\1.0\bin\perl.exe: *** unable to remap
f:\src\perl-5.8.8-trunkfix\perl-5.8.8\lib\auto\Fcntl\Fcntl.dll to same
address as parent -- 0x3D0000
5091622 [main] perl 10684 sync_with_child: child 5136(0x124) died before
initialization with status code 0x1
5091641 [main] perl 10684 sync_with_child: *** child state child loading
dlls
F:\Msys-got\1.0\bin\perl.exe: *** unable to remap
f:\src\perl-5.8.8-trunkfix\perl-5.8.8\lib\auto\Fcntl\Fcntl.dll to same
address as parent -- 0x3D0000
10154750 [main] perl 10684 sync_with_child: child 6068(0x138) died
before initialization with status code 0x1
10154770 [main] perl 10684 sync_with_child: *** child state child
loading dlls
F:\Msys-got\1.0\bin\perl.exe: *** unable to remap
f:\src\perl-5.8.8-trunkfix\perl-5.8.8\lib\auto\Fcntl\Fcntl.dll to same
address as parent -- 0x3D0000
15217449 [main] perl 10684 sync_with_child: child 13012(0x144) died
before initialization with status code 0x1
15217470 [main] perl 10684 sync_with_child: *** child state child
loading dlls
F:\Msys-got\1.0\bin\perl.exe: *** unable to remap
f:\src\perl-5.8.8-trunkfix\perl-5.8.8\lib\auto\Fcntl\Fcntl.dll to same
address as parent -- 0x410000
20279772 [main] perl 10684 sync_with_child: child 10004(0x150) died
before initialization with status code 0x1
20279791 [main] perl 10684 sync_with_child: *** child state child
loading dlls
Insecure dependency in `` while running with -T switch at op/taint.t
line 317.
# Looks like you planned 238 tests but ran 30.
dubious
     Test returned status 9 (wstat 2304, 0x900)
DIED. FAILED tests 1-3, 5, 31-238
     Failed 212/238 tests, 10.92% okay

lib/filefind-taint..........................dubious
     Test returned status 255 (wstat 65280, 0xff00)
DIED. FAILED tests 3-45
     Failed 43/45 tests, 4.44% okay

../ext/Cwd/t/cwd............................
#   Failed test in ../ext/Cwd/t/cwd.t at line 171.
#                   '/f/src/perl-5.8.8-trunkfix/perl-5.8.8/t/linktest'
#     doesn't match '(?-xism:t/_ptrslt_/_path_/_to_/_a_/_dir_$)'

#   Failed test in ../ext/Cwd/t/cwd.t at line 172.
#                   '/f/src/perl-5.8.8-trunkfix/perl-5.8.8/t/linktest'
#     doesn't match '(?-xism:t/_ptrslt_/_path_/_to_/_a_/_dir_$)'
# Looks like you failed 2 tests of 29.
dubious
     Test returned status 2 (wstat 512, 0x200)
DIED. FAILED tests 22-23
     Failed 2/29 tests, 93.10% okay (less 1 skipped test: 26 okay, 89.66%)
../ext/Cwd/t/taint..........................
#   Failed test 'its return value should be tainted'
#   in ../ext/Cwd/t/taint.t at line 31.

#   Failed test 'its return value should be tainted'
#   in ../ext/Cwd/t/taint.t at line 31.

#   Failed test 'its return value should be tainted'
#   in ../ext/Cwd/t/taint.t at line 31.

#   Failed test 'its return value should be tainted'
#   in ../ext/Cwd/t/taint.t at line 31.

#   Failed test 'its return value should be tainted'
#   in ../ext/Cwd/t/taint.t at line 31.

#   Failed test 'its return value should be tainted'
#   in ../ext/Cwd/t/taint.t at line 31.

#   Failed test 'its return value should be tainted'
#   in ../ext/Cwd/t/taint.t at line 31.

#   Failed test 'its return value should be tainted'
#   in ../ext/Cwd/t/taint.t at line 31.
# Looks like you failed 8 tests of 17.
dubious
     Test returned status 8 (wstat 2048, 0x800)
DIED. FAILED tests 2, 4, 6, 8, 10, 12, 14, 16
     Failed 8/17 tests, 52.94% okay

../ext/Socket/t/socketpair..................
#   Failed test 'socketpair (LEFT, RIGHT, AF_UNIX, SOCK_DGRAM, PF_UNSPEC)'
#   in ../ext/Socket/t/socketpair.t at line 176.
binmode() on closed filehandle LEFT at ../ext/Socket/t/socketpair.t line
181.
binmode() on closed filehandle RIGHT at ../ext/Socket/t/socketpair.t
line 182.
syswrite() on closed filehandle LEFT at ../ext/Socket/t/socketpair.t
line 187.

#   Failed test 'syswrite to left'
#   in ../ext/Socket/t/socketpair.t at line 187.
#          got: undef
#     expected: '6'
syswrite() on closed filehandle LEFT at ../ext/Socket/t/socketpair.t
line 187.

#   Failed test 'syswrite to left'
#   in ../ext/Socket/t/socketpair.t at line 187.
#          got: undef
#     expected: '6'
syswrite() on closed filehandle RIGHT at ../ext/Socket/t/socketpair.t
line 191.

#   Failed test 'syswrite to right'
#   in ../ext/Socket/t/socketpair.t at line 191.
#          got: undef
#     expected: '5'
syswrite() on closed filehandle RIGHT at ../ext/Socket/t/socketpair.t
line 191.

#   Failed test 'syswrite to right'
#   in ../ext/Socket/t/socketpair.t at line 191.
#          got: undef
#     expected: '6'
sysread() on closed filehandle LEFT at ../ext/Socket/t/socketpair.t line
199.

#   Failed test 'read on left'
#   in ../ext/Socket/t/socketpair.t at line 199.
#          got: undef
#     expected: '5'

#   Failed test 'content what we expected?'
#   in ../ext/Socket/t/socketpair.t at line 200.
#          got: ''
#     expected: 'perl '
sysread() on closed filehandle LEFT at ../ext/Socket/t/socketpair.t line
199.

#   Failed test 'read on left'
#   in ../ext/Socket/t/socketpair.t at line 199.
#          got: undef
#     expected: '6'

#   Failed test 'content what we expected?'
#   in ../ext/Socket/t/socketpair.t at line 200.
#          got: ''
#     expected: 'rules!'
sysread() on closed filehandle RIGHT at ../ext/Socket/t/socketpair.t
line 205.

#   Failed test 'read on right'
#   in ../ext/Socket/t/socketpair.t at line 205.
#          got: undef
#     expected: '6'

#   Failed test 'content what we expected?'
#   in ../ext/Socket/t/socketpair.t at line 206.
#          got: ''
#     expected: 'hello '
sysread() on closed filehandle RIGHT at ../ext/Socket/t/socketpair.t
line 205.

#   Failed test 'read on right'
#   in ../ext/Socket/t/socketpair.t at line 205.
#          got: undef
#     expected: '6'

#   Failed test 'content what we expected?'
#   in ../ext/Socket/t/socketpair.t at line 206.
#          got: ''
#     expected: 'world
# '
shutdown() on closed socket LEFT at ../ext/Socket/t/socketpair.t line 209.

#   Failed test 'shutdown left for writing'
#   in ../ext/Socket/t/socketpair.t at line 209.
sysread() on closed filehandle RIGHT at ../ext/Socket/t/socketpair.t
line 222.

#   Failed test 'alarm should have fired'
#   in ../ext/Socket/t/socketpair.t at line 224.
#          got: '0'
#     expected: '1'
syswrite() on closed filehandle RIGHT at ../ext/Socket/t/socketpair.t
line 232.

#   Failed test 'syswrite to right'
#   in ../ext/Socket/t/socketpair.t at line 232.
#          got: undef
#     expected: '1'
syswrite() on closed filehandle RIGHT at ../ext/Socket/t/socketpair.t
line 232.

#   Failed test 'syswrite to right'
#   in ../ext/Socket/t/socketpair.t at line 232.
#          got: undef
#     expected: '1'
sysread() on closed filehandle LEFT at ../ext/Socket/t/socketpair.t line
238.

#   Failed test 'read on left'
#   in ../ext/Socket/t/socketpair.t at line 238.
#          got: undef
#     expected: '1'

#   Failed test 'content what we expected?'
#   in ../ext/Socket/t/socketpair.t at line 239.
#          got: ''
#     expected: ' '
sysread() on closed filehandle LEFT at ../ext/Socket/t/socketpair.t line
238.

#   Failed test 'read on left'
#   in ../ext/Socket/t/socketpair.t at line 238.
#          got: undef
#     expected: '1'

#   Failed test 'content what we expected?'
#   in ../ext/Socket/t/socketpair.t at line 239.
#          got: ''
#     expected: ''

#   Failed test 'close left'
#   in ../ext/Socket/t/socketpair.t at line 242.

#   Failed test 'close right'
#   in ../ext/Socket/t/socketpair.t at line 243.
# Looks like you failed 23 tests of 45.
dubious
     Test returned status 23 (wstat 5888, 0x1700)
DIED. FAILED tests 22-35, 37-45
     Failed 23/45 tests, 48.89% okay

../lib/File/Copy............................
#   Failed test in ../lib/File/Copy.t at line 156.

#   Failed test in ../lib/File/Copy.t at line 158.
#                   ''
#     doesn't match '(?-xism:are identical)'

#   Failed test in ../lib/File/Copy.t at line 156.

#   Failed test in ../lib/File/Copy.t at line 158.
#                   ''
#     doesn't match '(?-xism:are identical)'
# Looks like you failed 4 tests of 60.
dubious
     Test returned status 4 (wstat 1024, 0x400)
DIED. FAILED tests 28-29, 55-56
     Failed 4/60 tests, 93.33% okay

../lib/File/Find/t/find.....................# Use of uninitialized value
in pattern match (m//) at ../lib/File/Find/t/find.t line 688.
rm: cannot lstat `for_find/dangling_dir': Permission denied
rm: cannot lstat `for_find/fa/faa/faa_sl': Permission denied
rm: cannot lstat `for_find/fb/fba': Permission denied
FAILED tests 26-28, 38-40, 51-53, 62-64, 80-82, 109-113, 170, 184, 186,
188, 200-227
     Failed 52/199 tests, 73.87% okay
../lib/File/Find/t/taint....................
#   Failed test 'Expected and found fa/fsl/fb_ord'
#   in ../lib/File/Find/t/taint.t at line 107.

#   Failed test 'Expected and found fa/fsl/fba'
#   in ../lib/File/Find/t/taint.t at line 107.

#   Failed test 'Expected and found fa/fsl/fba/fba_ord'
#   in ../lib/File/Find/t/taint.t at line 107.

#   Failed test 'Bad untaint pattern causes death in cwd (good)'
#   in ../lib/File/Find/t/taint.t at line 314.
#                   ''
#     doesn't match '(?-xism:insecure cwd)'

#   Failed test 'Cwd not untainted with bad pattern (good)'
#   in ../lib/File/Find/t/taint.t at line 384.
#                   ''
#     doesn't match '(?-xism:insecure cwd)'
rm: cannot lstat `for_find/fb/fba': Permission denied
# Looks like you planned 45 tests but ran 3 extra.
# Looks like you failed 5 tests of 48 run.
dubious
     Test returned status 5 (wstat 1280, 0x500)
DIED. FAILED tests 26-28, 32, 46-48
     Failed 7/45 tests, 84.44% okay

../lib/File/Spec/t/Spec.....................# Test 468 got:
"///../../..//a//b/../c" (../lib/File/Spec/t/Spec.t at line 672 fail #385)
#     Expected: "/a/b/../c"
(File::Spec::Epoc->canonpath('///../../..//./././a//b/.././c/././'))
#  ../lib/File/Spec/t/Spec.t line 672 is:     ok $got, $expected, $function;
FAILED test 468
     Failed 1/472 tests, 99.79% okay (less 83 skipped tests: 388 okay,
82.20%)

../lib/Net/Ping/t/450_service...............# Failed test 9 in
../lib/Net/Ping/t/450_service.t at line 84
#  ../lib/Net/Ping/t/450_service.t line 84 is: ok $p -> ping("127.0.0.1");
# Failed test 18 in ../lib/Net/Ping/t/450_service.t at line 143
#  ../lib/Net/Ping/t/450_service.t line 143 is: ok $p -> ack();
FAILED tests 9, 18
     Failed 2/26 tests, 92.31% okay

../lib/Test/t/multiline.....................FAILED test 2
     Failed 1/2 tests, 50.00% okay


------------------------------------------------------------------------------
What You Don't Know About Data Connectivity CAN Hurt You
This paper provides an overview of data connectivity, details
its effect on application quality, and explores various alternative
solutions. http://p.sf.net/sfu/progress-d2d
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe

envt.pl (780 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Msys-perl vs EBADF on fclose()

lrn-2
The next episode of everyone's favourite "Shaping up msys-perl" show!

After i concluded that perl is as good as i am ever going to make it,
i've decided to work a bit more on the packaging - so that it can be
shipped in its current state. Until now i've been promptly ignoring
CPAN, but that had to change, since both Msys and msysgit require some
extra perl modules that are not part of the default installation.
And first thing i've tried to install was Compression::Zlib.
Alas, CPAN refused to install it, pointing out that some testcases have
failed. I've looked them up. The first one to fail is zlib-generic.pl,
line 93:
ok $x->close ;

Banged my head against that one for some time. Eventually i've
discovered that fclose() in msys returns 0 (which is good) with errno ==
EBADF (which is bad) almost every time. I guess that code of most
programs is not paranoid enough to check errno after fclose(),
especially if fclose() itself returns 0, and that is why this bug was
not noticed until now. Or maybe i'm wrong. Also, perl will save and
restore errno by itself when calling fclose() with file descriptor
substituted to a bad one (perl wants to make CRT free the FILE stream,
while keeping the underlying FD intact), and that will mask the error
(intentionally, since perl expects EBADF in that case).
I've tried to rig perl to immediately close every FILE it opens, and
that doesn't seem to produce non-zero errno. Obviously, it's difficult
for me to track all the operations made on such stream between its
opening and closing, so i do not know what causes this bug.

I've made a hack that resets errno to 0 if fclose() returned 0 and
Compression::Zlib passed the test. Not sure how it will affect perl's
own testsuide, have not run it with this hack yet.

Also, i think i'd need an updated list of CPAN modules that should be
installed. The list in 5.6.1 CYGWIN BUNDLE might be redundant (i think
that some of these modules have made it into perl source tree; also,
i've found that Digest:MD5 is not only installed, but also used by perl
when it runs, making it impossible to upgrade it with CPAN).


------------------------------------------------------------------------------
What You Don't Know About Data Connectivity CAN Hurt You
This paper provides an overview of data connectivity, details
its effect on application quality, and explores various alternative
solutions. http://p.sf.net/sfu/progress-d2d
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Msys-perl vs EBADF on fclose()

Charles Wilson-2
On 3/7/2011 12:27 AM, LRN wrote:

> The next episode of everyone's favourite "Shaping up msys-perl" show!
>
> After i concluded that perl is as good as i am ever going to make it,
> i've decided to work a bit more on the packaging - so that it can be
> shipped in its current state. Until now i've been promptly ignoring
> CPAN, but that had to change, since both Msys and msysgit require some
> extra perl modules that are not part of the default installation.
> And first thing i've tried to install was Compression::Zlib.
> Alas, CPAN refused to install it, pointing out that some testcases have
> failed. I've looked them up. The first one to fail is zlib-generic.pl,
> line 93:
> ok $x->close ;
>
> Banged my head against that one for some time. Eventually i've
> discovered that fclose() in msys returns 0 (which is good) with errno ==
> EBADF (which is bad) almost every time.

This sounds like an MSYS bug.  E.g. fclose should return EOF on error,
and only then set errno:

http://pubs.opengroup.org/onlinepubs/009695399/functions/fclose.html
"Upon successful completion, fclose() shall return 0; otherwise, it
shall return EOF and set errno to indicate the error."

e.g. it should only set errno in the "otherwise" case -- e.g.
unsuccessful completion.

Instead of trying to work around the bug in perl, I'd suggest figuring
out why msys is doing the wrong thing, and fix that.  Do you have a
simple test case (C, not perl?) that demonstrates the erroneous behavior?

> Also, i think i'd need an updated list of CPAN modules that should be
> installed. The list in 5.6.1 CYGWIN BUNDLE might be redundant

I'd go with the list of bundled packages in the cygwin 5.8.8 releases:

exts="Win32API-File-0.1001 \
        Pod-Escapes-1.04 Pod-Simple-3.05 Test-Pod-1.26
Devel-Symdump-2.07 Pod-Coverage-0.18 Test-Pod-Coverage-1.08 \
        IO-Compress-Base-2.005 \
        Compress-Raw-Zlib-2.005 IO-Compress-Zlib-2.005 Compress-Zlib-2.005 \
        Compress-Raw-Bzip2-2.005 IO-Compress-Bzip2-2.005
Compress-Bzip2-2.09 \
        IO-Zlib-1.05    \
        IO-String-1.08  \
        MD5-2.03 \
        Archive-Tar-1.32 Archive-Zip-1.20 \
        Math-BigInt-FastCalc-0.15 \
        Term-ReadLine-Perl-1.0302 Term-ReadLine-Gnu-1.16 TermReadKey-2.30 \
        XML-NamespaceSupport-1.09 XML-SAX-0.15 XML-LibXML-Common-0.13
XML-LibXML-1.63 \
        XML-Parser-2.34 \
        Proc-ProcessTable-0.41 \
        File-Temp-0.18 YAML-0.62 \
        Config-Tiny-2.10 File-Copy-Recursive-0.33 IPC-Run3-0.037
Probe-Perl-0.01 \
        Tee-0.13 IO-CaptureOutput-1.03 File-pushd-0.99 File-HomeDir-0.65 \
        Digest-SHA-5.45 \
        Module-Signature-0.55 \
        URI-1.35 HTML-Tagset-3.10 HTML-Parser-3.56 libwww-perl-5.805 \
        CPAN-1.9102 \
        Test-Reporter-1.27 CPAN-Reporter-0.44 \
        Net-Telnet-3.03 \
        Module-ScanDeps-0.75 PAR-Dist-0.23 \
        ExtUtils-CBuilder-0.19 ExtUtils-ParseXS-2.18 Regexp-Common-2.120 \
          version-0.7203 podlators-2.0.5 Pod-Readme-0.09
Module-Build-0.2808 \
        B-Generate-1.09 PadWalker-1.5 Alias-2.32"

Plus, handled as a special case, perl-${ver}-ext-Win32CORE.tar.bz2.

In fact, I'd take a very close look at the contents and build script
included in cygwin's perl-5.8.8-4-src.tar.bz2 package:

http://mirrors.kernel.org/sourceware/cygwin/release-legacy/perl/

Note that the cygwin-perl build script automatically builds and installs
[1] each of the exts.

The cygwin package ALSO includes a number of cygwin-specific patches for
both the perl core and some of these exts, that with suitable munging,
might also be useful on MSYS.

(FYI, I would probably hesitate to "update" these external packages to
their latest current release, and would lean towards using, exactly,
these older versions.  (A) because in some cases they have accompanying
patches for cygwin, and (B) it's possible newer versions require newer
perl...)

[1] using install_vendor. Because modules installed into site_perl (as
opposed to vendor_perl) have higher precedence, end users will be able
to update to newer versions if they so desire using CPAN.  This is a
worry on cygwin, given that setup.exe-installed stuff is usually
writable only by Administrator.  MSys-perl probably doesn't need to
worry about this (as we don't really have much in the way of access
control on installed files), so we COULD install bundled extensions into
site_perl instead...but then, mingw-get might become confused if the
installed fileset doesn't match the manifest when it comes time to
update.  So, I'd stick with install_vendor.

--
Chuck


------------------------------------------------------------------------------
What You Don't Know About Data Connectivity CAN Hurt You
This paper provides an overview of data connectivity, details
its effect on application quality, and explores various alternative
solutions. http://p.sf.net/sfu/progress-d2d
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Msys-perl vs EBADF on fclose()

Benjamin Riefenstahl
In reply to this post by lrn-2
Hi,

LRN writes:

> [...]
> failed. I've looked them up. The first one to fail is zlib-generic.pl,
> line 93:
> ok $x->close ;
>
> Banged my head against that one for some time. Eventually i've
> discovered that fclose() in msys returns 0 (which is good) with errno
> == EBADF (which is bad) almost every time. I guess that code of most
> programs is not paranoid enough to check errno after fclose(),
> especially if fclose() itself returns 0, and that is why this bug was

I don't think that this is a bug in fclose.  In general, errno can only
be used after a function fails.  IOW any function can set errno to any
value, that doesn't mean anything unless the function than declares that
it failed, via its return value or by other means.  There are exceptions
but AFAICT fclose is not such an exception.

benny

------------------------------------------------------------------------------
What You Don't Know About Data Connectivity CAN Hurt You
This paper provides an overview of data connectivity, details
its effect on application quality, and explores various alternative
solutions. http://p.sf.net/sfu/progress-d2d
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Msys-perl vs EBADF on fclose()

lrn-2
In reply to this post by Charles Wilson-2
On 07.03.2011 17:59, Charles Wilson wrote:

> On 3/7/2011 12:27 AM, LRN wrote:
>> The next episode of everyone's favourite "Shaping up msys-perl" show!
>>
>> After i concluded that perl is as good as i am ever going to make it,
>> i've decided to work a bit more on the packaging - so that it can be
>> shipped in its current state. Until now i've been promptly ignoring
>> CPAN, but that had to change, since both Msys and msysgit require some
>> extra perl modules that are not part of the default installation.
>> And first thing i've tried to install was Compression::Zlib.
>> Alas, CPAN refused to install it, pointing out that some testcases have
>> failed. I've looked them up. The first one to fail is zlib-generic.pl,
>> line 93:
>> ok $x->close ;
>>
>> Banged my head against that one for some time. Eventually i've
>> discovered that fclose() in msys returns 0 (which is good) with errno ==
>> EBADF (which is bad) almost every time.
> This sounds like an MSYS bug.  E.g. fclose should return EOF on error,
> and only then set errno:
>
> http://pubs.opengroup.org/onlinepubs/009695399/functions/fclose.html
> "Upon successful completion, fclose() shall return 0; otherwise, it
> shall return EOF and set errno to indicate the error."
>
> e.g. it should only set errno in the "otherwise" case -- e.g.
> unsuccessful completion.
>
> Instead of trying to work around the bug in perl, I'd suggest figuring
> out why msys is doing the wrong thing, and fix that.
This is fclose from msysCORE:

int
_DEFUN (fclose, (fp),
     register FILE * fp)
{
   int r;

   if (fp == NULL)
     return (0);            /* on NULL */

   CHECK_INIT (fp);

   if (fp->_flags == 0)        /* not open! */
     return (0);
   r = fp->_flags & __SWR ? fflush (fp) : 0;
   if (fp->_close != NULL && (*fp->_close) (fp->_cookie) < 0)
     r = EOF;
   if (fp->_flags & __SMBF)
     _free_r (fp->_data, (char *) fp->_bf._base);
   if (HASUB (fp))
     FREEUB (fp);
   if (HASLB (fp))
     FREELB (fp);
   fp->_flags = 0;        /* release this FILE for reuse */
   return (r);
}

AFAICS, it does not set errno by itself, msvcrt's close() does that.
>    Do you have a
> simple test case (C, not perl?) that demonstrates the erroneous behavior?
This is the testcase:

#include <stdio.h>
#include <errno.h>

int main (int argc, char **argv)
{
   FILE *f;
   int n;
   printf ("errno == %d\n", errno);
   f = fopen ("tmpfile.txt", "wb");
   printf ("Opened %p, errno == %d\n", f, errno);
   n = fprintf (f, "Hello World!\n");
   printf ("Wrote %d bytes into %p, errno == %d\n", n, f, errno);
   n = fclose (f);
   printf ("Closed %p, got %d, errno == %d\n", f, n, errno);
   return 0;
}

To compile:
gcc -Wall main.c -o msysfclose.exe

I get:
$ msysfclose.exe
errno == 0
Opened 0xa01079c, errno == 0
Wrote 13 bytes into 0xa01079c, errno == 0
Closed 0xa01079c, got 0, errno == 9

When compiled with mingw i get correct results:
$ mingwfclose.exe
errno == 0
Opened 75A32960, errno == 0
Wrote 13 bytes into 75A32960, errno == 0
Closed 75A32960, got 0, errno == 0


------------------------------------------------------------------------------
What You Don't Know About Data Connectivity CAN Hurt You
This paper provides an overview of data connectivity, details
its effect on application quality, and explores various alternative
solutions. http://p.sf.net/sfu/progress-d2d
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Msys-perl vs EBADF on fclose()

lrn-2
On 07.03.2011 19:01, LRN wrote:

> On 07.03.2011 17:59, Charles Wilson wrote:
>> On 3/7/2011 12:27 AM, LRN wrote:
>>> The next episode of everyone's favourite "Shaping up msys-perl" show!
>>>
>>> After i concluded that perl is as good as i am ever going to make it,
>>> i've decided to work a bit more on the packaging - so that it can be
>>> shipped in its current state. Until now i've been promptly ignoring
>>> CPAN, but that had to change, since both Msys and msysgit require some
>>> extra perl modules that are not part of the default installation.
>>> And first thing i've tried to install was Compression::Zlib.
>>> Alas, CPAN refused to install it, pointing out that some testcases have
>>> failed. I've looked them up. The first one to fail is zlib-generic.pl,
>>> line 93:
>>> ok $x->close ;
>>>
>>> Banged my head against that one for some time. Eventually i've
>>> discovered that fclose() in msys returns 0 (which is good) with
>>> errno ==
>>> EBADF (which is bad) almost every time.
>> This sounds like an MSYS bug.  E.g. fclose should return EOF on error,
>> and only then set errno:
>>
>> http://pubs.opengroup.org/onlinepubs/009695399/functions/fclose.html
>> "Upon successful completion, fclose() shall return 0; otherwise, it
>> shall return EOF and set errno to indicate the error."
>>
>> e.g. it should only set errno in the "otherwise" case -- e.g.
>> unsuccessful completion.
>>
>> Instead of trying to work around the bug in perl, I'd suggest figuring
>> out why msys is doing the wrong thing, and fix that.
> [snip]
>
I've modified fclose.c, here's the part that differs:
   if (fp->_close != NULL)
   {
     int msvcrt_r;
     printf ("fclose() -> _close:%p (%p), errno == %d\n", fp->_close,
fp->_cookie, errno);
     msvcrt_r = (*fp->_close) (fp->_cookie);
     printf ("fclose() <- _close:%p (%p) returned %d, errno == %d\n",
fp->_close, fp->_cookie, msvcrt_r, errno);
     if (msvcrt_r < 0)
     {
       printf ("fclose() will return %d\n", EOF);
       r = EOF;
     }
   }

I've also reduced the testcase a bit, removing all statements between
fopen() and fclose(). With a debug version of msys and with the testcase
built against msys running under gdb i get this:

fclose() -> _close:0x608916a6 (0xa011e64), errno == 0
warning: MsYs00000058 10 4948240 [main] msysfclose2 6336
fhandler_console::write: 54 = write_console (,..54)

warning: MsYs0000004A 80 4950019 [main] msysfclose2 6336 _write: 54 =
write (1, 0xA011A50, 54)

warning: MsYs00000036 10 4951529 [main] msysfclose2 6336 _close: close (3)

warning: MsYs00000047 10 4952787 [main] msysfclose2 6336
fhandler_base::close: handle 0x114

warning: MsYs00000083 10 4954233 [main] msysfclose2 6336
seterrno_from_win_error:
/f/src/msysCORE-1.0.16-1/source/winsup/cygwin/syscalls.cc:970 errno 6

warning: MsYs00000058 10 4956782 [main] msysfclose2 6336
geterrno_from_win_error: windows error 6 == errno 9

warning: MsYs0000003A 10 4958722 [main] msysfclose2 6336 _close: 0 =
close (3)

warning: MsYs00000045 80 4962820 [main] msysfclose2 6336 _write: write
(1, 0xA011A50, 65)

warning: MsYs00000049 40 4964247 [main] msysfclose2 6336
fhandler_console::write: A011A50, 65

warning: MsYs00000052 40 4965629 [main] msysfclose2 6336
fhandler_console::write: at 102(f) state is 1

fclose() <- _close:0x608916a6 (0xa011e64) returned 0, errno == 9

If msys to be believed, the errno comes from a failing
FlushFileBuffers() call in fsync() in syscalls.cc

------------------------------------------------------------------------------
What You Don't Know About Data Connectivity CAN Hurt You
This paper provides an overview of data connectivity, details
its effect on application quality, and explores various alternative
solutions. http://p.sf.net/sfu/progress-d2d
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Msys-perl vs EBADF on fclose()

lrn-2
On 07.03.2011 20:56, LRN wrote:

> On 07.03.2011 19:01, LRN wrote:
>> On 07.03.2011 17:59, Charles Wilson wrote:
>>> On 3/7/2011 12:27 AM, LRN wrote:
>>>> The next episode of everyone's favourite "Shaping up msys-perl" show!
>>>>
>>>> After i concluded that perl is as good as i am ever going to make it,
>>>> i've decided to work a bit more on the packaging - so that it can be
>>>> shipped in its current state. Until now i've been promptly ignoring
>>>> CPAN, but that had to change, since both Msys and msysgit require some
>>>> extra perl modules that are not part of the default installation.
>>>> And first thing i've tried to install was Compression::Zlib.
>>>> Alas, CPAN refused to install it, pointing out that some testcases
>>>> have
>>>> failed. I've looked them up. The first one to fail is zlib-generic.pl,
>>>> line 93:
>>>> ok $x->close ;
>>>>
>>>> Banged my head against that one for some time. Eventually i've
>>>> discovered that fclose() in msys returns 0 (which is good) with
>>>> errno ==
>>>> EBADF (which is bad) almost every time.
>>> This sounds like an MSYS bug.  E.g. fclose should return EOF on error,
>>> and only then set errno:
>>>
>>> http://pubs.opengroup.org/onlinepubs/009695399/functions/fclose.html
>>> "Upon successful completion, fclose() shall return 0; otherwise, it
>>> shall return EOF and set errno to indicate the error."
>>>
>>> e.g. it should only set errno in the "otherwise" case -- e.g.
>>> unsuccessful completion.
>>>
>>> Instead of trying to work around the bug in perl, I'd suggest figuring
>>> out why msys is doing the wrong thing, and fix that.
>> [snip]
>>
> [snip]
>
> If msys to be believed, the errno comes from a failing
> FlushFileBuffers() call in fsync() in syscalls.cc

This is _close() from syscalls.cc:

extern "C" int
_close (int fd)
{
   int res;
   sigframe thisframe (mainthread);

   syscall_printf ("close (%d)", fd);

   MALLOC_CHECK;
   if (cygheap->fdtab.not_open (fd))
     {
       debug_printf ("handle %d not open", fd);
       set_errno (EBADF);
       res = -1;
     }
   else
     {
       SetResourceLock (LOCK_FD_LIST,WRITE_LOCK|READ_LOCK," close");
       res = cygheap->fdtab[fd]->close ();
       fsync(fd);
       cygheap->fdtab.release (fd);
       ReleaseResourceLock (LOCK_FD_LIST,WRITE_LOCK|READ_LOCK," close");
     }

   syscall_printf ("%d = close (%d)", res, fd);
   MALLOC_CHECK;
   return res;
}

After tracing it in gdb i've discovered that this line:
res = cygheap->fdtab[fd]->close ();
Ultimately calls CloseHandle(h).
After that anything fsync(fd) does with the same value of (h) leads to
an error.

The extra fsync(fd); appears somewhere between 1.0.10 and 1.0.11
Positioning it before `res = cygheap->fdtab[fd]->close ();' seems to
solve the bug (the testcase reports errno == 0 after fclose()).

------------------------------------------------------------------------------
What You Don't Know About Data Connectivity CAN Hurt You
This paper provides an overview of data connectivity, details
its effect on application quality, and explores various alternative
solutions. http://p.sf.net/sfu/progress-d2d
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Msys-perl vs EBADF on fclose()

Keith Marshall
In reply to this post by Charles Wilson-2
On 07/03/11 14:59, Charles Wilson wrote:
>> Banged my head against that one for some time. Eventually i've
>> discovered that fclose() in msys returns 0 (which is good) with errno ==
>>  EBADF (which is bad) almost every time.

Why is this bad?  If fclose() has returned zere, there is no error, and
you have no right to expect errno to have any particular value; if you
expect it to be zero, (or anything else, for that matter), then you are
relying on undefined behaviour, and that's a bug in your program.

> This sounds like an MSYS bug.

I  don't think so.

> E.g. fclose should return EOF on error, and only then set errno:

Yes, if fclose() returns EOF, errno should be set appropriately, but if
fclose() has returned zero, then errno could be anything; its value is
undefined in this case, and it is a bug to expect otherwise.

> Instead of trying to work around the bug in perl, I'd suggest figuring
> out why msys is doing the wrong thing, and fix that.

But MSYS *isn't* doing the wrong thing; it isn't *required* to do
anything in particular; it is perl (or the OP) doing the wrong thing, by
testing for a particular outcome from undefined behaviour.

--
Regards,
Keith.

------------------------------------------------------------------------------
What You Don't Know About Data Connectivity CAN Hurt You
This paper provides an overview of data connectivity, details
its effect on application quality, and explores various alternative
solutions. http://p.sf.net/sfu/progress-d2d
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Msys-perl vs EBADF on fclose()

Keith Marshall
In reply to this post by lrn-2
On 07/03/11 19:56, LRN wrote:
> The extra fsync(fd); appears somewhere between 1.0.10 and 1.0.11
> Positioning it before `res = cygheap->fdtab[fd]->close ();' seems to
> solve the bug (the testcase reports errno == 0 after fclose()).

But errno isn't required to be zero after a successful fclose(); the
only bug I see here is in your flawed expectations -- you expect a
particular outcome from undefined behaviour!

--
Regards,
Keith.

------------------------------------------------------------------------------
What You Don't Know About Data Connectivity CAN Hurt You
This paper provides an overview of data connectivity, details
its effect on application quality, and explores various alternative
solutions. http://p.sf.net/sfu/progress-d2d
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Msys-perl vs EBADF on fclose()

Keith Marshall
In reply to this post by lrn-2
On 07/03/11 05:27, LRN wrote:
> I guess that code of most programs is not paranoid enough to check
> errno after fclose(), especially if fclose() itself returns 0,

Huh?  If fclose() has returned zero, there is no error, and the value of
errno is undefined.  It is very, very wrong to expect it to be zero, (or
any other value).  Your paranoid program, expecting a particular errno
value after a function reports success, is just plain broken.

--
Regards,
Keith.

------------------------------------------------------------------------------
What You Don't Know About Data Connectivity CAN Hurt You
This paper provides an overview of data connectivity, details
its effect on application quality, and explores various alternative
solutions. http://p.sf.net/sfu/progress-d2d
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Msys-perl vs EBADF on fclose()

Earnie Boyd
In reply to this post by Keith Marshall
Keith Marshall wrote:
> On 07/03/11 19:56, LRN wrote:
>> The extra fsync(fd); appears somewhere between 1.0.10 and 1.0.11
>> Positioning it before `res = cygheap->fdtab[fd]->close ();' seems to
>> solve the bug (the testcase reports errno == 0 after fclose()).
>
> But errno isn't required to be zero after a successful fclose(); the
> only bug I see here is in your flawed expectations -- you expect a
> particular outcome from undefined behaviour!
>

But you may set errno to zero before calling fclose and on successful
fclose the variable should remain zero unless some other function was
called and set it.  The setting of errno is based always on error and
never on success by the standard libraries regardless of the standard
function being called.

--
Earnie
-- http://www.for-my-kids.com

------------------------------------------------------------------------------
What You Don't Know About Data Connectivity CAN Hurt You
This paper provides an overview of data connectivity, details
its effect on application quality, and explores various alternative
solutions. http://p.sf.net/sfu/progress-d2d
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Msys-perl vs EBADF on fclose()

Charles Wilson-2
In reply to this post by Keith Marshall
On 3/7/2011 3:28 PM, Keith Marshall wrote:
> On 07/03/11 19:56, LRN wrote:
>> The extra fsync(fd); appears somewhere between 1.0.10 and 1.0.11
>> Positioning it before `res = cygheap->fdtab[fd]->close ();' seems to
>> solve the bug (the testcase reports errno == 0 after fclose()).
>
> But errno isn't required to be zero after a successful fclose(); the
> only bug I see here is in your flawed expectations -- you expect a
> particular outcome from undefined behaviour!

I disagree. I realize this is arguing from silence, but the spec does
NOT say that errno can be set when success -- it only says that errno
SHALL be set on failure.  By implication, this means it should not be
*explicitly* set on success.  It may (or may not) be explicitly cleared
by the successful call to fclose, depending on whether the runtime wants
to allow failure codes arising from earlier failed syscalls to persist
or not -- but that's not what's going on in this test case.

Remember that we're talking about MSys here, not msvcrt directly.  MSys,
like cygwin, should give POSIX behavior whenever possible.  cygwin has
recently (last several years) adopted the view that, when the POSIX
specification is unclear, and there is no overriding reason (*) to
choose otherwise, behavior should match GNU/Linux.

On Fedora 14 (2.6.35.10-74.fc14.x86_64) with glibc-2.13-1.x86_64 and
gcc-4.5.1-4.fc14.x86_64, I get this behavior:

errno == 0
Opened 0xb8f010, errno == 0
Wrote 13 bytes into 0xb8f010, errno == 0
Closed 0xb8f010, got 0, errno == 0


Now, I haven't had a chance to look into the Msys guts yet, but if it
really is as simple as moving an fsync() -- and that change has no other
ill effects -- then I don't see why we wouldn't do that.

This is, of course, a *separate* question as to why perl is checking
errno after a "successful" call to fclose in the first place.


(*) such as "it would really complicate the internal cygwin (msys) code,
for only minimal gain"

--
Chuck

------------------------------------------------------------------------------
What You Don't Know About Data Connectivity CAN Hurt You
This paper provides an overview of data connectivity, details
its effect on application quality, and explores various alternative
solutions. http://p.sf.net/sfu/progress-d2d
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Msys-perl vs EBADF on fclose()

Keith Marshall
In reply to this post by Earnie Boyd
On 07/03/11 20:41, Earnie wrote:
> But you may set errno to zero before calling fclose and on
> successful fclose the variable should remain zero unless some other
> function was called and set it.

You may, but you have no guarantee that it will remain so.

> The setting of errno is based always on error and never on success by
> the standard libraries regardless of the standard function being
> called.

Exactly.  The system function you called may have called some other
function internally.  That function may have set errno; the one you
called may have handled the exception, such that it can then return
successfully, and the value of errno is then undefined.

The only circumstance under which you may legitimately test errno, and
expect a defined value, is *after* you have tested the return value from
a function you've called, *and* verified an *unsuccessful* outcome.

--
Regards,
Keith.

------------------------------------------------------------------------------
What You Don't Know About Data Connectivity CAN Hurt You
This paper provides an overview of data connectivity, details
its effect on application quality, and explores various alternative
solutions. http://p.sf.net/sfu/progress-d2d
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Msys-perl vs EBADF on fclose()

lrn-2
In reply to this post by Keith Marshall
On 07.03.2011 23:28, Keith Marshall wrote:
> On 07/03/11 19:56, LRN wrote:
>> The extra fsync(fd); appears somewhere between 1.0.10 and 1.0.11
>> Positioning it before `res = cygheap->fdtab[fd]->close ();' seems to
>> solve the bug (the testcase reports errno == 0 after fclose()).
> But errno isn't required to be zero after a successful fclose(); the
> only bug I see here is in your flawed expectations -- you expect a
> particular outcome from undefined behaviour!
>
s/your/perl's or Compress::Zlib's/
s/you/perl or Compress::Zlib/

There IS an ACTUAL bug in there - an attempt to fsync() a closed file. I
expect that it will be fixed, which will automagically fix errno to
remain 0 (or whatever it was before the call). Thus, the point is
largely moot. In the worst case i still can fix perl itself. Everything
else is just bikeshedding.

By the way, i am still unsure of WHY perl freaks out - i mean, fclose()
DOES return 0, and as you've said, perl has no reason to check errno -
and in the function that i've seen it indeed doesn't; if it does, that
happens somewhere between perl's *close() and zlib test calling
$x->close(), because according to perldoc the `ok' function that is used
to check the teststep only checks the return value.

------------------------------------------------------------------------------
What You Don't Know About Data Connectivity CAN Hurt You
This paper provides an overview of data connectivity, details
its effect on application quality, and explores various alternative
solutions. http://p.sf.net/sfu/progress-d2d
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Msys-perl vs EBADF on fclose()

Keith Marshall
In reply to this post by Charles Wilson-2
On 07/03/11 20:51, Charles Wilson wrote:

> On 3/7/2011 3:28 PM, Keith Marshall wrote:
>> On 07/03/11 19:56, LRN wrote:
>>> The extra fsync(fd); appears somewhere between 1.0.10 and 1.0.11
>>> Positioning it before `res = cygheap->fdtab[fd]->close ();' seems
>>> to solve the bug (the testcase reports errno == 0 after
>>> fclose()).
>>
>> But errno isn't required to be zero after a successful fclose();
>> the only bug I see here is in your flawed expectations -- you
>> expect a particular outcome from undefined behaviour!
>
> I disagree.

Then we must agree to disagree; however, I don't see disagreement in
your follow up...

> I realize this is arguing from silence, but the spec does NOT say
> that errno can be set when success -- it only says that errno SHALL
> be set on failure.  By implication, this means it should not be
> *explicitly* set on success.

Exactly so; it *may* be set to anything at all, or untouched.

> It may (or may not) be explicitly cleared by the successful call to
> fclose,

Again, exactly so; it *may* be cleared, but it isn't *required* that it
be; thus, it *is* a bug to expect that it is.

> Remember that we're talking about MSys here, not msvcrt directly.
> MSys, like cygwin, should give POSIX behavior whenever possible.
> cygwin has recently (last several years) adopted the view that, when
>  the POSIX specification is unclear, and there is no overriding
> reason (*) to choose otherwise, behavior should match GNU/Linux.
>
> On Fedora 14 (2.6.35.10-74.fc14.x86_64) with glibc-2.13-1.x86_64 and
> gcc-4.5.1-4.fc14.x86_64, I get this behavior:

Where the standard doesn't require any particular behaviour, what Fedora
does is irrelevant.

> errno == 0 Opened 0xb8f010, errno == 0 Wrote 13 bytes into 0xb8f010,
>  errno == 0 Closed 0xb8f010, got 0, errno == 0
>
>
> Now, I haven't had a chance to look into the Msys guts yet, but if
> it really is as simple as moving an fsync() -- and that change has no
>  other ill effects -- then I don't see why we wouldn't do that.

That is entirely down to you and Cesar to choose; you *may* do so, but
you aren't *required* to.

> This is, of course, a *separate* question as to why perl is checking
> errno after a "successful" call to fclose in the first place.

Exactly my point; after a successful call, the standard doesn't require
that errno be set to *any* particular value; it *is* a bug (in perl) to
expect that it is, and that this bug may not be caught on some systems
doesn't make it any less a bug.

--
Regards,
Keith.

------------------------------------------------------------------------------
What You Don't Know About Data Connectivity CAN Hurt You
This paper provides an overview of data connectivity, details
its effect on application quality, and explores various alternative
solutions. http://p.sf.net/sfu/progress-d2d
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Msys-perl vs EBADF on fclose()

Earnie Boyd
In reply to this post by lrn-2
LRN wrote:
> There IS an ACTUAL bug in there - an attempt to fsync() a closed file.

You know where the patch track resides. :D

> I expect that it will be fixed, which will automagically fix errno to
> remain 0 (or whatever it was before the call). Thus, the point is
> largely moot. In the worst case i still can fix perl itself. Everything
> else is just bikeshedding.
>
> By the way, i am still unsure of WHY perl freaks out - i mean, fclose()
> DOES return 0, and as you've said, perl has no reason to check errno -
> and in the function that i've seen it indeed doesn't; if it does, that
> happens somewhere between perl's *close() and zlib test calling
> $x->close(), because according to perldoc the `ok' function that is used
> to check the teststep only checks the return value.

So this is a case where two bugs doesn't make it correct but had the
fsync on closed fd bug not been made the 2nd bug of perl checking errno
on a non-error would not have risen its ugly head.  Yes, both need fixed.

--
Earnie
-- http://www.for-my-kids.com

------------------------------------------------------------------------------
What You Don't Know About Data Connectivity CAN Hurt You
This paper provides an overview of data connectivity, details
its effect on application quality, and explores various alternative
solutions. http://p.sf.net/sfu/progress-d2d
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Msys-perl vs EBADF on fclose()

Charles Wilson-2
In reply to this post by Keith Marshall
On 3/7/2011 4:45 PM, Keith Marshall wrote:
> On 07/03/11 20:51, Charles Wilson wrote:
>> I disagree.
>
> Then we must agree to disagree; however, I don't see disagreement in
> your follow up...

It boils down to this...

>> MSys, like cygwin, should give POSIX behavior whenever possible.
>> cygwin has recently (last several years) adopted the view that, when
>>  the POSIX specification is unclear, and there is no overriding
>> reason (*) to choose otherwise, behavior should match GNU/Linux.

...I think, like Cygwin, MSys should act like linux, where POSIX allows
and it's feasible (*) to do so, and manpower exists to make it happen.

(*) for various definitions of "feasible"

>> On Fedora 14 (2.6.35.10-74.fc14.x86_64) with glibc-2.13-1.x86_64 and
>> gcc-4.5.1-4.fc14.x86_64, I get this behavior:
>
> Where the standard doesn't require any particular behaviour, what Fedora
> does is irrelevant.

Without a policy as stated as above, you're right. WITH such a policy,
then it boils down to "where feasible and manpower exists".

However, LRN is correct about one thing: it is a BUG if, in MSys's own
code, it does this:

   _fclose(fh); // calls msvcrt's fclose impl
   _fsync(fh);  // and then calls msvcrt's fsync impl

That's just wrong. (For clarity: I haven't yet verified that this
accurately describes what's going on in the guts of Msys)

--
Chuck

------------------------------------------------------------------------------
What You Don't Know About Data Connectivity CAN Hurt You
This paper provides an overview of data connectivity, details
its effect on application quality, and explores various alternative
solutions. http://p.sf.net/sfu/progress-d2d
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Msys-perl vs EBADF on fclose()

dhoyt
In reply to this post by Charles Wilson-2
> I'd go with the list of bundled packages in the cygwin 5.8.8 releases:
>
> exts="Win32API-File-0.1001 \
>         Pod-Escapes-1.04 Pod-Simple-3.05 Test-Pod-1.26
> Devel-Symdump-2.07 Pod-Coverage-0.18 Test-Pod-Coverage-1.08 \
>         IO-Compress-Base-2.005 \
>         Compress-Raw-Zlib-2.005 IO-Compress-Zlib-2.005 Compress-Zlib-2.005 \
>         Compress-Raw-Bzip2-2.005 IO-Compress-Bzip2-2.005
> Compress-Bzip2-2.09 \
>         IO-Zlib-1.05    \
>         IO-String-1.08  \
>         MD5-2.03 \
>         Archive-Tar-1.32 Archive-Zip-1.20 \
>         Math-BigInt-FastCalc-0.15 \
>         Term-ReadLine-Perl-1.0302 Term-ReadLine-Gnu-1.16 TermReadKey-2.30 \
>         XML-NamespaceSupport-1.09 XML-SAX-0.15 XML-LibXML-Common-0.13
> XML-LibXML-1.63 \
>         XML-Parser-2.34 \
>         Proc-ProcessTable-0.41 \
>         File-Temp-0.18 YAML-0.62 \
>         Config-Tiny-2.10 File-Copy-Recursive-0.33 IPC-Run3-0.037
> Probe-Perl-0.01 \
>         Tee-0.13 IO-CaptureOutput-1.03 File-pushd-0.99 File-HomeDir-0.65 \
>         Digest-SHA-5.45 \
>         Module-Signature-0.55 \
>         URI-1.35 HTML-Tagset-3.10 HTML-Parser-3.56 libwww-perl-5.805 \
>         CPAN-1.9102 \
>         Test-Reporter-1.27 CPAN-Reporter-0.44 \
>         Net-Telnet-3.03 \
>         Module-ScanDeps-0.75 PAR-Dist-0.23 \
>         ExtUtils-CBuilder-0.19 ExtUtils-ParseXS-2.18 Regexp-Common-2.120 \
>           version-0.7203 podlators-2.0.5 Pod-Readme-0.09
> Module-Build-0.2808 \
>         B-Generate-1.09 PadWalker-1.5 Alias-2.32"

+1 to the XML-Parser package so intltool will work correctly. Having makemaker would be nice as well.

------------------------------------------------------------------------------
What You Don't Know About Data Connectivity CAN Hurt You
This paper provides an overview of data connectivity, details
its effect on application quality, and explores various alternative
solutions. http://p.sf.net/sfu/progress-d2d
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Msys-perl vs EBADF on fclose()

lrn-2
In reply to this post by Earnie Boyd
On 08.03.2011 1:53, Earnie wrote:

> LRN wrote:
>> There IS an ACTUAL bug in there - an attempt to fsync() a closed file.
> You know where the patch track resides. :D
>
>> I expect that it will be fixed, which will automagically fix errno to
>> remain 0 (or whatever it was before the call). Thus, the point is
>> largely moot. In the worst case i still can fix perl itself. Everything
>> else is just bikeshedding.
>>
>> By the way, i am still unsure of WHY perl freaks out - i mean, fclose()
>> DOES return 0, and as you've said, perl has no reason to check errno -
>> and in the function that i've seen it indeed doesn't; if it does, that
>> happens somewhere between perl's *close() and zlib test calling
>> $x->close(), because according to perldoc the `ok' function that is used
>> to check the teststep only checks the return value.
> So this is a case where two bugs doesn't make it correct but had the
> fsync on closed fd bug not been made the 2nd bug of perl checking errno
> on a non-error would not have risen its ugly head.  Yes, both need fixed.
>
As i have said, i do not know what is going wrong in perl and where.
Debugging perl is a chore for me, to be honest. So i will be content
with fixing this on msys side first and foremost, since it is now known.

On 08.03.2011 1:55, Charles Wilson wrote:
> However, LRN is correct about one thing: it is a BUG if, in MSys's own
> code, it does this:
>
>     _fclose(fh); // calls msvcrt's fclose impl
>     _fsync(fh);  // and then calls msvcrt's fsync impl
>
> That's just wrong. (For clarity: I haven't yet verified that this
> accurately describes what's going on in the guts of Msys)
>
Well, my previous theories were a bit misleading, as Msys re-implements
syscalls and [to some extent?] CRT. Which means that msvcrt (as far as i
can see) plays [almost] no part in this, WinAPI is used instead. msys'
fclose() calls filehandler::close (or something like that, i don't
remember), which eventually calls CloseHandle() on the handle. Then
msys' fclose() calls msys' fsync(), which is implemented as a call to
FlushFileBuffers() on the handle. Both functions end up calling WinAPI
on msys level (msvcrt would have called WinAPI eventually too, but we
would not have known it for sure). I've checked the value of the handle
at debug time - it is the same for both CloseHandle() and
FlushFileBuffers().

It's easily trackable in a debug version of msys when debugging my
testcase under gdb.

Msys does some magic along the way too, but it is mostly related to
keeping track of FDs and whatnot.

------------------------------------------------------------------------------
What You Don't Know About Data Connectivity CAN Hurt You
This paper provides an overview of data connectivity, details
its effect on application quality, and explores various alternative
solutions. http://p.sf.net/sfu/progress-d2d
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Msys-perl vs EBADF on fclose()

lrn-2
In reply to this post by lrn-2
On 08.03.2011 0:43, LRN wrote:

> On 07.03.2011 23:28, Keith Marshall wrote:
>> On 07/03/11 19:56, LRN wrote:
>>> The extra fsync(fd); appears somewhere between 1.0.10 and 1.0.11
>>> Positioning it before `res = cygheap->fdtab[fd]->close ();' seems to
>>> solve the bug (the testcase reports errno == 0 after fclose()).
>> But errno isn't required to be zero after a successful fclose(); the
>> only bug I see here is in your flawed expectations -- you expect a
>> particular outcome from undefined behaviour!
>>
> s/your/perl's or Compress::Zlib's/
> s/you/perl or Compress::Zlib/
>
> There IS an ACTUAL bug in there - an attempt to fsync() a closed file.
> I expect that it will be fixed, which will automagically fix errno to
> remain 0 (or whatever it was before the call). Thus, the point is
> largely moot. In the worst case i still can fix perl itself.
> Everything else is just bikeshedding.
A quick update: After testing the new msys-1.0.dll i've discovered that
it affects some applications in a way i do not understand. The result is
that perl configure script freezes at random points.
To investigate that i've compiled 3 fresh version of msysCORE - the
"stock" one (no modifications), the "moved" one (fsync() call is moved
before close()) and the "commented" one (fsync() call is simply
commented out). Tried to configure perl with each one of them. Only the
"moved" one makes it freeze.
So i am led to think that the best (most compatible, less disturbing)
way to fix this is to simply comment-out fsync() rather than move it.
Because at the moment (in the "stock" version) fsync's only contribution
to fclose() is updating errno to 9, FlushFileBuffers() has no effect.
Moving it before close() changes the situation - it actually flushes
before closing. My guess is that this flush is not something that
applications (or maybe the other parts of msys - i've no idea) expect,
and that breaks things. Commenting out fsync() call will remove the
unneeded errno (fixing the original symptom for me) without altering the
behaviour.

------------------------------------------------------------------------------
What You Don't Know About Data Connectivity CAN Hurt You
This paper provides an overview of data connectivity, details
its effect on application quality, and explores various alternative
solutions. http://p.sf.net/sfu/progress-d2d
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
123