Time for native MSYS?

classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

Time for native MSYS?

Chris Wilson-4
Hi all,

This may be a stupid question, please forgive me if so (and I'd appreciate
any pointers to help me learn why).

I've been using Cygwin and their port of MinGW GCC for many years to build
native executables on Windows. It's doable but the performance is a
killer, especially build times. I suspect that Cygwin's fork emulation
is required for every invocation of gcc, cp, cpp, rm, bash, etc...

I've been thinking about using MSYS, but my initial experiments told me
that it doesn't provide nearly as much drop-in compatibility for configure
scripts as Cygwin does; and I suspected it would be just as slow because
MSYS components run under the MSYS dll (an old fork of the Cygwin DLL) in
any case.

Is there any point in trying to port the more common utilities away from
the MSYS DLL to native implementations? E.g. I can imagine that using
native bash, gcc, rm, etc. might be much faster. Does that make sense?
Would it be difficult? Perhaps the less commonly-used utilities could
remain under MSYS to ease the porting effort?

Thanks in advance,

Chris.
--
_____ __     _
\  __/ / ,__(_)_  | Chris Wilson <[hidden email]> Cambs UK |
/ (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer |
\__/_/_/_//_/___/ | We are GNU : free your mind & your software |

------------------------------------------------------------------------------
Free Software Download: Index, Search & Analyze Logs and other IT data in
Real-Time with Splunk. Collect, index and harness all the fast moving IT data
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Time for native MSYS?

Keith Marshall-2
On 27/02/11 21:25, Chris Wilson wrote:
> I've been thinking about using MSYS, but my initial experiments told
> me that it doesn't provide nearly as much drop-in compatibility for
> configure scripts as Cygwin does;

That's not been my experience.  Any GCS conforming configure script
should run equally well under MSYS and Cygwin sh.exe; perhaps if you
need an esoteric tool, Cygwin is more likely to provide it.

> and I suspected it would be just as slow because MSYS components run
> under the MSYS dll (an old fork of the Cygwin DLL) in any case.

I find MSYS to be slightly faster than Cygwin, but it's still slow,
which is why I prefer to use a cross-compiler hosted on GNU/Linux; it's
an order of magnitude faster.

> Is there any point in trying to port the more common utilities away
> from the MSYS DLL to native implementations? E.g. I can imagine that
> using native bash, gcc, rm, etc. might be much faster. Does that make
> sense?  Would it be difficult?  Perhaps the less commonly-used
> utilities could remain under MSYS to ease the porting effort?

Well, our GCC and binutils tool chain is already 100% native.  Many
of the other tools are already available, as native Win32 applications,
from the GnuWin32 project; I've tried some of them, but I've always had
better success using the MSYS variants.  You are welcome to try them;
YMMV.  One stumbling block: last time I looked, I don't think they
offered a Unixy shell; rather a show stopper, if you want to run
a Bourne shell configure script.

--
Regards,
Keith.

------------------------------------------------------------------------------
Free Software Download: Index, Search & Analyze Logs and other IT data in
Real-Time with Splunk. Collect, index and harness all the fast moving IT data
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Time for native MSYS?

Chris Wilson-4
Hi Keith and all,

On Sun, 27 Feb 2011, Keith Marshall wrote:
> On 27/02/11 21:25, Chris Wilson wrote:
>> I've been thinking about using MSYS, but my initial experiments told
>> me that it doesn't provide nearly as much drop-in compatibility for
>> configure scripts as Cygwin does;
>
> That's not been my experience.  Any GCS conforming configure script
> should run equally well under MSYS and Cygwin sh.exe; perhaps if you
> need an esoteric tool, Cygwin is more likely to provide it.

The project I'm working on requires OpenSSL and PCRE, both of which have
their own non-standard configure mechanisms, and I seem to remember
having difficulty getting them to work with MSYS (it was a long time ago).
If someone knows that either of these works, please let me know (and if
possible, how to do it) :)

> I find MSYS to be slightly faster than Cygwin, but it's still slow,
> which is why I prefer to use a cross-compiler hosted on GNU/Linux; it's
> an order of magnitude faster.

I agree, but when it comes to testing, Wine doesn't cut it; and this
project has several configure tests that rely on being able to execute
programs, not just compile them, which also makes cross-compiling more
difficult. Finally there are no RPMs for MinGW packages for CentOS, which
is one of my main development platforms. (They now exist for Fedora 8,
which I guess might help if I had time to recompile gcc, g++, etc.)

>> Is there any point in trying to port the more common utilities away
>> from the MSYS DLL to native implementations? E.g. I can imagine that
>> using native bash, gcc, rm, etc. might be much faster. Does that make
>> sense?  Would it be difficult?  Perhaps the less commonly-used
>> utilities could remain under MSYS to ease the porting effort?
>
> Well, our GCC and binutils tool chain is already 100% native.

Does that mean that they avoid the startup overhead of loading
msys-1.0.dll and forking CPP? If so then perhaps I'm barking up the wrong
tree.

> Many of the other tools are already available, as native Win32
> applications, from the GnuWin32 project; I've tried some of them, but
> I've always had better success using the MSYS variants.  You are welcome
> to try them; YMMV.  One stumbling block: last time I looked, I don't
> think they offered a Unixy shell; rather a show stopper, if you want to
> run a Bourne shell configure script.

Yes, I gather that bash doesn't run natively, but zsh apparently does, and
it might be "close enough" (I've heard a lot of people recommending zsh).

As far as the other tools go, I suspect that they don't do path
translation in the same way as MSYS does, and this might cause problems
for configure scripts.

Cheers, Chris.
--
_____ __     _
\  __/ / ,__(_)_  | Chris Wilson <[hidden email]> Cambs UK |
/ (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer |
\__/_/_/_//_/___/ | We are GNU : free your mind & your software |

------------------------------------------------------------------------------
Free Software Download: Index, Search & Analyze Logs and other IT data in
Real-Time with Splunk. Collect, index and harness all the fast moving IT data
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Time for native MSYS?

Charles Wilson-2
In reply to this post by Keith Marshall-2
On 2/27/2011 5:21 PM, Keith Marshall wrote:
> I find MSYS to be slightly faster than Cygwin, but it's still slow,
> which is why I prefer to use a cross-compiler hosted on GNU/Linux; it's
> an order of magnitude faster.

In general, MSYS should be faster than Cygwin -- because (a) it doesn't
do as much access checking, and (b) all the new posix support features
that have been added to cygwin over the last seven years, such as
i18n/widechar support, impose a runtime cost.

I have heard claims that running a linux guest in a virtual machine on a
win32 host, and using a mingw-target cross compiler IN the linux guest,
is actually faster than using either MinGW/MSYS or Cygwin...

--
Chuck


------------------------------------------------------------------------------
Free Software Download: Index, Search & Analyze Logs and other IT data in
Real-Time with Splunk. Collect, index and harness all the fast moving IT data
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Time for native MSYS?

Greg Chicares
In reply to this post by Chris Wilson-4
On 2011-02-27 23:18Z, Chris Wilson wrote:
> On Sun, 27 Feb 2011, Keith Marshall wrote:
>> On 27/02/11 21:25, Chris Wilson wrote:
[...]
>>> Is there any point in trying to port the more common utilities away
>>> from the MSYS DLL to native implementations? E.g. I can imagine that
>>> using native bash, gcc, rm, etc. might be much faster. Does that make
>>> sense?  Would it be difficult?  Perhaps the less commonly-used
>>> utilities could remain under MSYS to ease the porting effort?

I looked into that a few years ago, and it seemed too difficult for me.

Besides, MSYS needs versions that recognize the MSYS filesystem so that
they can interpret commands like
  rm ~/foo.bar
so I suppose they have to be linked to the MSYS dll. If a particular
utility is especially slow, and you can identify an optimization like
replacing fork() and exec() with spawn() and push that change upstream,
then everyone would benefit.

>> Well, our GCC and binutils tool chain is already 100% native.
>
> Does that mean that they avoid the startup overhead of loading
> msys-1.0.dll and forking CPP?

Using a mingw.org version of gcc-3.4.5:

$cygcheck /MinGW_/bin/gcc
C:\opt\lmi\MinGW-20090203\bin\gcc.exe
  C:\WINDOWS\system32\KERNEL32.dll
    C:\WINDOWS\system32\ntdll.dll
  C:\WINDOWS\system32\msvcrt.dll

I don't know exactly how it invokes cpp, but I doubt it calls fork().

>> Many of the other tools are already available, as native Win32
>> applications, from the GnuWin32 project; I've tried some of them, but
>> I've always had better success using the MSYS variants.  You are welcome
>> to try them; YMMV.  One stumbling block: last time I looked, I don't
>> think they offered a Unixy shell; rather a show stopper, if you want to
>> run a Bourne shell configure script.
>
> Yes, I gather that bash doesn't run natively, but zsh apparently does, and
> it might be "close enough" (I've heard a lot of people recommending zsh).

I used to use Amol Deshpande's old native port of zsh, but IIRC it was
zsh-3.0.5, which is outmoded--e.g., it doesn't provide printf(1). It
lives on, unmaintained, in this 2003 file:
  http://unxutils.sourceforge.net/zsh.zip
which IIRC is zsh-3.0.8: not much more modern. It's "resurrected" here:
  http://zsh-nt.sourceforge.net/
but that's still 3.0.8, a 2000-05-16 release. Unless it's actively
maintained, you're up a creek when you see "fork failed". I dropped it
years ago in favor of MSYS and Cygwin for reliability reasons.

------------------------------------------------------------------------------
Free Software Download: Index, Search & Analyze Logs and other IT data in
Real-Time with Splunk. Collect, index and harness all the fast moving IT data
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Time for native MSYS?

lrn-2
In reply to this post by Charles Wilson-2
On 28.02.2011 2:31, Charles Wilson wrote:

> On 2/27/2011 5:21 PM, Keith Marshall wrote:
>> I find MSYS to be slightly faster than Cygwin, but it's still slow,
>> which is why I prefer to use a cross-compiler hosted on GNU/Linux; it's
>> an order of magnitude faster.
> I have heard claims that running a linux guest in a virtual machine on a
> win32 host, and using a mingw-target cross compiler IN the linux guest,
> is actually faster than using either MinGW/MSYS or Cygwin...
>
> --
> Chuck
>
In that case you can implement something like MinWG (Minimalist Windows
for GNU) and connect its Windows and *nix parts (one running natively,
another - in VM) with sockets. If you handle the shared filesystem and
I/O redirection correctly, you'd be able to run PE binaries on Windows,
while everything else will remain in the VM.


...and no, i haven't been smoking anything...

------------------------------------------------------------------------------
Free Software Download: Index, Search & Analyze Logs and other IT data in
Real-Time with Splunk. Collect, index and harness all the fast moving IT data
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Time for native MSYS?

Michael T.
In reply to this post by Charles Wilson-2

> I have heard claims that running a linux guest in a virtual machine on a
> win32 host, and using a mingw-target cross compiler IN the linux guest,
> is actually faster than using either MinGW/MSYS or Cygwin...

Interesting.. What particular virtual machine? Is a benchmark test available?

>
> --
> Chuck
>
>
> ------------------------------------------------------------------------------
> Free Software Download: Index, Search & Analyze Logs and other IT data in
> Real-Time with Splunk. Collect, index and harness all the fast moving IT data
> generated by your applications, servers and devices whether physical, virtual
> or in the cloud. Deliver compliance at lower cost and gain new business
> insights. http://p.sf.net/sfu/splunk-dev2dev 
> _______________________________________________
> MinGW-users mailing list
> [hidden email]
>
> This list observes the Etiquette found at
> http://www.mingw.org/Mailing_Lists.
> We ask that you be polite and do the same.  Disregard for the list etiquette may
> cause your account to be moderated.
>
> _______________________________________________
> You may change your MinGW Account Options or unsubscribe at:
> https://lists.sourceforge.net/lists/listinfo/mingw-users
> Also: mailto:[hidden email]?subject=unsubscribe
>
>
>

------------------------------------------------------------------------------
Free Software Download: Index, Search & Analyze Logs and other IT data in
Real-Time with Splunk. Collect, index and harness all the fast moving IT data
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Time for native MSYS?

Earnie Boyd
In reply to this post by Chris Wilson-4
Chris Wilson wrote:
> Hi Keith and all,
>> Well, our GCC and binutils tool chain is already 100% native.
>
> Does that mean that they avoid the startup overhead of loading
> msys-1.0.dll and forking CPP? If so then perhaps I'm barking up the wrong
> tree.

The compiler and linker set are linked to msvcrt.dll and not
msys-1.0.dll.  If you use MSYS then MSYS will need to convert the path
strings before spawning the native application.  This is still faster
than Cygwin because the SYMLINK code has been removed since SYMLINK
doesn't work on Windows and native msvcrt.dll doesn't understand
shortcuts.  Also MSYS doesn't use the registry so you can store it on a
USB disk, use it on any computer and not leave a footprint.

--
Earnie
-- http://www.for-my-kids.com

------------------------------------------------------------------------------
Free Software Download: Index, Search & Analyze Logs and other IT data in
Real-Time with Splunk. Collect, index and harness all the fast moving IT data
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Time for native MSYS?

Roumen Petrov
In reply to this post by Chris Wilson-4
Chris Wilson wrote:
> Hi all,
>
>    
[snip]

ash from unmaintained  pw32 projects and bash from mingw (msys) work
slow under windows emulation - about 10 times slower then native bash.
No idea how ash is build . It work under wine but same time crash.
msys bash work under wine since ... sorry I forgot wine version that
resolve bash crash.
msys dash still crash under wine.

Sorry I could not help as I'm not familiar with internals of above shell
and why all of them are so slow.
I guess that native performance is related to the windows process
management - it is slow.


Regards,
Roumen


------------------------------------------------------------------------------
Free Software Download: Index, Search & Analyze Logs and other IT data in
Real-Time with Splunk. Collect, index and harness all the fast moving IT data
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Time for native MSYS?

Charles Wilson-2
In reply to this post by Michael T.
On 2/28/2011 4:28 AM, Michael T. wrote:
>
>> I have heard claims that running a linux guest in a virtual machine on a
>> win32 host, and using a mingw-target cross compiler IN the linux guest,
>> is actually faster than using either MinGW/MSYS or Cygwin...
>
> Interesting.. What particular virtual machine? Is a benchmark test available?

Don't know.  They were "claims" not "double blind scientific studies".
At this point I don't even recall where I first heard/read them.

But...if somebody actually DID do some tests like this on identical
hardware for an apples-to-apples comparison it'd sure be interesting.

--
Chuck

------------------------------------------------------------------------------
Free Software Download: Index, Search & Analyze Logs and other IT data in
Real-Time with Splunk. Collect, index and harness all the fast moving IT data
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Time for native MSYS?

d3x0r
On Mon, Feb 28, 2011 at 6:16 PM, Charles Wilson
<[hidden email]> wrote:

> On 2/28/2011 4:28 AM, Michael T. wrote:
>>
>>> I have heard claims that running a linux guest in a virtual machine on a
>>> win32 host, and using a mingw-target cross compiler IN the linux guest,
>>> is actually faster than using either MinGW/MSYS or Cygwin...
>>
>> Interesting.. What particular virtual machine? Is a benchmark test available?
>
> Don't know.  They were "claims" not "double blind scientific studies".
> At this point I don't even recall where I first heard/read them.
>
> But...if somebody actually DID do some tests like this on identical
> hardware for an apples-to-apples comparison it'd sure be interesting.
>

Heh that is a fact; for a long time I mainted code for linux/windows,
I just use make and gcc basically though; nothing as complex as
configure.  But certainly cygwin is the slowest like 66% speed unless
you use -mno-cygwin; msys I have no exp.; gcc and make under windows
is slower than gcc and make under linux in a vmware box on the same
computer.  The linux box SEEMS at  least 200% faster IIRC, but sadly I
don't have specific benchmarks anymore;  I could start the same
compile using basicaly the same compilers under windows and the vmware
linux and the linux box will finish ahead 100% of the time.
especially something like cmake that invokes processes repeatedly.

> --
> Chuck
>
> ------------------------------------------------------------------------------
> Free Software Download: Index, Search & Analyze Logs and other IT data in
> Real-Time with Splunk. Collect, index and harness all the fast moving IT data
> generated by your applications, servers and devices whether physical, virtual
> or in the cloud. Deliver compliance at lower cost and gain new business
> insights. http://p.sf.net/sfu/splunk-dev2dev
> _______________________________________________
> MinGW-users mailing list
> [hidden email]
>
> This list observes the Etiquette found at
> http://www.mingw.org/Mailing_Lists.
> We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.
>
> _______________________________________________
> You may change your MinGW Account Options or unsubscribe at:
> https://lists.sourceforge.net/lists/listinfo/mingw-users
> Also: mailto:[hidden email]?subject=unsubscribe
>

------------------------------------------------------------------------------
Free Software Download: Index, Search & Analyze Logs and other IT data in
Real-Time with Splunk. Collect, index and harness all the fast moving IT data
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Time for native MSYS?

Andy Koppe
In reply to this post by Earnie Boyd
On 28 February 2011 15:12, Earnie wrote:

> Chris Wilson wrote:
>> Hi Keith and all,
>>> Well, our GCC and binutils tool chain is already 100% native.
>>
>> Does that mean that they avoid the startup overhead of loading
>> msys-1.0.dll and forking CPP? If so then perhaps I'm barking up the wrong
>> tree.
>
> The compiler and linker set are linked to msvcrt.dll and not
> msys-1.0.dll.  If you use MSYS then MSYS will need to convert the path
> strings before spawning the native application.  This is still faster
> than Cygwin because the SYMLINK code has been removed

I doubt that symlink support has much to do with any speed difference.
The biggee is fork().

> Also MSYS doesn't use the registry so you can store it on a
> USB disk, use it on any computer and not leave a footprint.

Same for Cygwin 1.7. The DLL assumes that the directory above its own
is the POSIX root directory and reads mount points from /etc/fstab.
Not sure how that's relevant to this thread though.

Andy

------------------------------------------------------------------------------
Free Software Download: Index, Search & Analyze Logs and other IT data in
Real-Time with Splunk. Collect, index and harness all the fast moving IT data
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

OpenSSL configure [was: Re: Time for native MSYS?]

Albrecht Schlosser
In reply to this post by Chris Wilson-4
On 28.02.2011 00:18, Chris Wilson wrote:

> The project I'm working on requires OpenSSL and PCRE, both of which have
> their own non-standard configure mechanisms, and I seem to remember
> having difficulty getting them to work with MSYS (it was a long time ago).
> If someone knows that either of these works, please let me know (and if
> possible, how to do it) :)

Hmm, I don't know if you really meant OpenSSL to run *with* MSYS (i.e.
with the MSYS dll), or native. I'm sure you know that there is a current
msys-openssl package to download, but that's not up-to-date.

I needed OpenSSL for native Windows (w/o MSYS), and it turned out that
recent versions (1.0.0c) work OOTB. I used:

$ ./config no-threads --prefix=... \
   --openssldir='C:/mingw/msys/1.0/...'

Note the path in Windows notation "C:..." with forward slashes to avoid
more escaping. I also configured w/o any options, and it worked...

HTH
Albrecht


------------------------------------------------------------------------------
Free Software Download: Index, Search & Analyze Logs and other IT data in
Real-Time with Splunk. Collect, index and harness all the fast moving IT data
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Time for native MSYS?

Earnie Boyd
In reply to this post by Andy Koppe
Andy Koppe wrote:

> On 28 February 2011 15:12, Earnie wrote:
>> Chris Wilson wrote:
>>> Hi Keith and all,
>>>> Well, our GCC and binutils tool chain is already 100% native.
>>>
>>> Does that mean that they avoid the startup overhead of loading
>>> msys-1.0.dll and forking CPP? If so then perhaps I'm barking up the wrong
>>> tree.
>>
>> The compiler and linker set are linked to msvcrt.dll and not
>> msys-1.0.dll.  If you use MSYS then MSYS will need to convert the path
>> strings before spawning the native application.  This is still faster
>> than Cygwin because the SYMLINK code has been removed
>
> I doubt that symlink support has much to do with any speed difference.
> The biggee is fork().
>

You can doubt all you want but your doubt isn't based on anything
factual.  In Cygwin every file is opened and read looking for a symlink
signature.  If a symlink signature is found the file referenced is then
opened and read looking for a symlink signature.  You don't think that
is slow, fine, but I will disagree.

>> Also MSYS doesn't use the registry so you can store it on a
>> USB disk, use it on any computer and not leave a footprint.
>
> Same for Cygwin 1.7. The DLL assumes that the directory above its own
> is the POSIX root directory and reads mount points from /etc/fstab.
> Not sure how that's relevant to this thread though.

I am well aware of the change to use /etc/fstab in Cygwin 1.7.  But
where do you think the idea came from?  I happen to know that back in
the day when I forked Cygwin to create MSYS that cgf was dead set
against the idea of a file based mount registry.  It wasn't until many
years later that Cygwin followed suit after MSYS had proven it worked
well.  However, IIRC, Cygwin still uses the registry for other things so
using it will still leave a footprint.  The relevance has to do with the
fact that the discussion is MSYS (viz Cygwin).

--
Earnie
-- http://www.for-my-kids.com

------------------------------------------------------------------------------
Free Software Download: Index, Search & Analyze Logs and other IT data in
Real-Time with Splunk. Collect, index and harness all the fast moving IT data
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Time for native MSYS?

Charles Wilson-2
On 3/1/2011 7:46 AM, Earnie wrote:
> Andy Koppe wrote:
>> I doubt that symlink support has much to do with any speed difference.
>> The biggee is fork().
>
> You can doubt all you want but your doubt isn't based on anything
> factual.  In Cygwin every file is opened and read looking for a symlink

Yes (*), this is another time sync. 'Course, there's an open mingw
ticket to add symlink support (NTFS only, Vista+?) for MSYS, using
windows' support for the thing they CALL a symbolic link.  IMO its
semantics aren't an exact match for unix symlinks, but whatever.

Not sure if incorporating that feature will slow down MSYS' file access
to the degree cygwin's support for its version does.

(*) There have been some optimizations IIRC where the file is only
opened and read, if it has the system bit set, which helps somewhat (and
you get the info about the system bit "for free" when you read the
parent direntry).

> I am well aware of the change to use /etc/fstab in Cygwin 1.7.  But
> where do you think the idea came from?  I happen to know that back in
> the day when I forked Cygwin to create MSYS that cgf was dead set
> against the idea of a file based mount registry.  It wasn't until many
> years later that Cygwin followed suit after MSYS had proven it worked
> well.

MSYS was used as a data point: "see, it works fine over /there/".

However, Earnie's summary skips over one factoid: for years, cygwin devs
believed that the "entanglement" between copies of cygwin installed in
different locations -- and thus, a common registry-based, mechanism for
determining the mount table -- was a GOOD thing.

It allowed cygwin devs to detect if an end user was (erroneously) trying
to use applications with a mixture of different versions of cygwin. And,
that there was only ONE windows directory whos unixy path could be
expressed as "/bin" regardless of how many different installations of
the cygwin dll existed.  With MSYS, and now with cygwin, each individual
cygwin.dll has it's own /bin: whatever dir the DLL is in.


E.g. some 3PP provides cygwin-1.3.15 + bash, AND sets the global PATH,
to support $my-custom-application. This can cause the "real" cygwin
1.Much.Newer installation to fail in mysterious ways, so being able to
detect this was considered good.

The new behavior -- and MSYS's -- can fail if
        (a) you have a msys-dependent (cygwin-dependent) binary installed in a
directory OTHER than the /bin for that MSYS dll.
        (b) your app uses a new entry point provided only with newer versions
of the msys (cygwin) dll
        (c) your $PATH is structured so that the /bin containing the OLD msys
(cygwin) dll precedes the one containing the NEW msys (cygwin) dll

%PATH%=C:\msys-old\1.0\bin:<other stuff>

In this case, when you try to run C:\msys-new\1.0\local\bin\foo.exe, the
MSYS dll in C:\msys-old\1.0\bin\ will be loaded into memory. But since
the old msys dll doesn't provide _my_new_function(), and foo.exe needs
it, foo.exe will fail with a 0xc0000005 popup warning box "This app has
failed to start....".


In cygwin, this is particularly bad, because cygwins have for years
SUPPRESSED that particular warning box (it causes too many problems
during automated execution of test suites and other things), a cygwin
user who runs into this problem will just see:

My-CMD.EXE-prompt:
> foo.exe

My-CMD.EXE-prompt:
>

What fun.


In fact, even in modern cygwins you can revert to the "old" behavior --
but I forget how and am too lazy to look it up.  Which means instead of
the "nothing happened" experience, you get an informative warning about
multiple versions of cygwin installed.



Really, it wasn't entirely (or even mostly) about the registry-based or
file-based mount table. There are lots of globally shared data
structures that MSYS (and now, new-cygwin) had to restructure so that
they would NOT be shared between multiple installations of the msys
(cygwin) DLL...

> However, IIRC, Cygwin still uses the registry for other things so
> using it will still leave a footprint.

It /can/ use the registry and read values if they already exist, but
IIRC it will no longer set any registry keys on a virgin system.

--
Chuck

------------------------------------------------------------------------------
Free Software Download: Index, Search & Analyze Logs and other IT data in
Real-Time with Splunk. Collect, index and harness all the fast moving IT data
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Time for native MSYS?

Charles Wilson-2
In reply to this post by Earnie Boyd
On 3/1/2011 7:46 AM, Earnie wrote:
> Andy Koppe wrote:
>> I doubt that symlink support has much to do with any speed
>> difference. The biggee is fork().
>>
> You can doubt all you want but your doubt isn't based on anything
> factual.  In Cygwin every file is opened and read looking for a
> symlink
Yes (*), this is another time sync. 'Course, there's an open mingw
ticket to add symlink support (NTFS only, Vista+?) for MSYS, using
windows' support for the thing they CALL a symbolic link.  IMO its
semantics aren't an exact match for unix symlinks, but whatever.

Not sure if incorporating that feature will slow down MSYS' file access
to the degree cygwin's support for its version does.

(*) There have been some optimizations IIRC where the file is only
opened and read, if it has the system bit set, which helps somewhat (and
you get the info about the system bit "for free" when you read the
parent direntry).

> I am well aware of the change to use /etc/fstab in Cygwin 1.7.  But
> where do you think the idea came from?  I happen to know that back in
> the day when I forked Cygwin to create MSYS that cgf was dead set
> against the idea of a file based mount registry.  It wasn't until many
> years later that Cygwin followed suit after MSYS had proven it worked
> well.

MSYS was used as a data point: "see, it works fine over /there/".

However, Earnie's summary skips over one factoid: for years, cygwin devs
believed that the "entanglement" between copies of cygwin installed in
different locations -- and thus, a common registry-based, mechanism for
determining the mount table -- was a GOOD thing.

It allowed cygwin devs to detect if an end user was (erroneously) trying
to use applications with a mixture of different versions of cygwin. And,
that there was only ONE windows directory whos unixy path could be
expressed as "/bin" regardless of how many different installations of
the cygwin dll existed.  With MSYS, and now with cygwin, each individual
cygwin.dll has it's own /bin: whatever dir the DLL is in.


E.g. some 3PP provides cygwin-1.3.15 + bash, AND sets the global PATH,
to support $my-custom-application. This can cause the "real" cygwin
1.Much.Newer installation to fail in mysterious ways, so being able to
detect this was considered good.

The new behavior -- and MSYS's -- can fail if
        (a) you have a msys-dependent (cygwin-dependent) binary installed in a
directory OTHER than the /bin for that MSYS dll.
        (b) your app uses a new entry point provided only with newer versions
of the msys (cygwin) dll
        (c) your $PATH is structured so that the /bin containing the OLD msys
(cygwin) dll precedes the one containing the NEW msys (cygwin) dll

%PATH%=C:\msys-old\1.0\bin:<other stuff>

In this case, when you try to run C:\msys-new\1.0\local\bin\foo.exe, the
MSYS dll in C:\msys-old\1.0\bin\ will be loaded into memory. But since
the old msys dll doesn't provide _my_new_function(), and foo.exe needs
it, foo.exe will fail with a 0xc0000005 popup warning box "This app has
failed to start....".


In cygwin, this is particularly bad, because cygwins have for years
SUPPRESSED that particular warning box (it causes too many problems
during automated execution of test suites and other things), a cygwin
user who runs into this problem will just see:

My-CMD.EXE-prompt:
> foo.exe
My-CMD.EXE-prompt:
>
What fun.


In fact, even in modern cygwins you can revert to the "old" behavior --
but I forget how and am too lazy to look it up.  Which means instead of
the "nothing happened" experience, you get an informative warning about
multiple versions of cygwin installed.



Really, it wasn't entirely (or even mostly) about the registry-based or
file-based mount table. There are lots of globally shared data
structures that MSYS (and now, new-cygwin) had to restructure so that
they would NOT be shared between multiple installations of the msys
(cygwin) DLL...

> However, IIRC, Cygwin still uses the registry for other things so
> using it will still leave a footprint.

It /can/ use the registry and read values if they already exist, but
IIRC it will no longer set any registry keys on a virgin system.

--
Chuck


------------------------------------------------------------------------------
Free Software Download: Index, Search & Analyze Logs and other IT data in
Real-Time with Splunk. Collect, index and harness all the fast moving IT data
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Time for native MSYS?

Andy Koppe
In reply to this post by Earnie Boyd
On 1 March 2011 12:46, Earnie wrote:

> Andy Koppe wrote:
>> On 28 February 2011 15:12, Earnie wrote:
>>> The compiler and linker set are linked to msvcrt.dll and not
>>> msys-1.0.dll.  If you use MSYS then MSYS will need to convert the path
>>> strings before spawning the native application.  This is still faster
>>> than Cygwin because the SYMLINK code has been removed
>>
>> I doubt that symlink support has much to do with any speed difference.
>> The biggee is fork().
>>
> You can doubt all you want but your doubt isn't based on anything
> factual.  In Cygwin every file is opened and read looking for a symlink
> signature.

Symlinks are marked with the 'System' attribute, and only files with
that attribute set have to be opened to look for the symlink
signature. Attributes are part of directory entries, which have to be
read anyway when traversing a path, so there should be no additional
I/O in the normal case where the 'System' attribute isn't set.


>>> Also MSYS doesn't use the registry so you can store it on a
>>> USB disk, use it on any computer and not leave a footprint.
>>
>> Same for Cygwin 1.7. The DLL assumes that the directory above its own
>> is the POSIX root directory and reads mount points from /etc/fstab.
>> Not sure how that's relevant to this thread though.
>
> I am well aware of the change to use /etc/fstab in Cygwin 1.7.  But
> where do you think the idea came from?  I happen to know that back in
> the day when I forked Cygwin to create MSYS that cgf was dead set
> against the idea of a file based mount registry.  It wasn't until many
> years later that Cygwin followed suit after MSYS had proven it worked
> well.

Thanks for the history lesson. Still, this thread was specifically
about performance differences, not yet another general MSYS v Cygwin
discussion.


> However, IIRC, Cygwin still uses the registry for other things so
> using it will still leave a footprint.

Cygwin setup.exe stores the last install location in the registry, but
this isn't relevant when Cygwin is already installed on a USB disk.
(And I expect mingw-get-inst touches the registry too.)

The Cygwin 1.7 DLL, when initialized, stores a record of its own
location at HKLM/SOFTWARE/Cygwin/Installations (or under HKCU if
running as a limited user). This is used to help debug problems due to
mixed installations.

(Multiple Cygwin 1.7s can be used at the same time, but only processes
using the same DLL interact correctly. I see the approach for keeping
the DLLs apart was also pioneered by MSYS.)

If that Installations key really is a problem for someone, it can
easily be removed with regedit or Cygwin's own regtool. Then again, if
someone's that paranoid about leaving footprints in the registry, they
probably shouldn't attach a USB disk in the first place.

Andy

------------------------------------------------------------------------------
Free Software Download: Index, Search & Analyze Logs and other IT data in
Real-Time with Splunk. Collect, index and harness all the fast moving IT data
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe