Re: rsync fails with files over 2GB

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: rsync fails with files over 2GB

Dan Raymond
> Not a compiler problem. A 32-bit gcc can use 64-bit integers.
>
> The problem is that MSYS defines off_t as a 32-bit integer [1].  
> Because off_t has a sign bit, the maximum file size is 2GB - 1.  
> There is no way to fix this without breaking compatibility in MSYS:  
> one who changes off_t to 64-bit would also need to recompile all  
> programs that use MSYS.

> I can rsync a 2.2G file with MSYS2  
> <http://sourceforge.net/projects/msys2/>, so MSYS2 must have fixed  
> this.
> --George Koehler
>
> [1] See 'typedef long off_t;' in
> http://sourceforge.net/p/mingw/msys-runtime/ci/master/tree/newlib/libc/include/sys/types.h

It has been a while since this thread was started.  Has there been any  
effort to improve large file support in MSYS?  Also, I took a look at  
MSYS2 and I couldn't find rsync.exe.  George, was there an additional  
package you installed to get that binary?


------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: rsync fails with files over 2GB

マーズ
Can't be helped. 32-bit arch limit. Did you know malloc will
return bad_alloc if you try to allocate above 1 GByte?

[hidden email] wrote:

>> Not a compiler problem. A 32-bit gcc can use 64-bit
integers.
>>
>> The problem is that MSYS defines off_t as a 32-bit integer
[1].
>> Because off_t has a sign bit, the maximum file size is 2GB
- 1.
>> There is no way to fix this without breaking compatibility
in MSYS:
>> one who changes off_t to 64-bit would also need to
recompile all
>> programs that use MSYS.
>
>> I can rsync a 2.2G file with MSYS2
>> <http://sourceforge.net/projects/msys2/>, so MSYS2 must
have fixed
>> this.
>> --George Koehler
>>
>> [1] See 'typedef long off_t;' in
>> http://sourceforge.net/p/mingw/msys-
runtime/ci/master/tree/newlib/libc/include/sys/types.h
>
> It has been a while since this thread was started.  Has
there been any
> effort to improve large file support in MSYS?  Also, I took
a look at
> MSYS2 and I couldn't find rsync.exe.  George, was there an
additional
> package you installed to get that binary?
>
>
>
------------------------------------------------------------------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT
Server
> from Actuate! Instantly Supercharge Your Business Reports
and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App
Integration & more
> Get technology previously reserved for billion-dollar
corporations, FREE
>
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
> _______________________________________________
> MinGW-users mailing list
> MinGW-
[hidden email]
>
> This list observes the Etiquette found at
> http://www.mingw.org/Mailing_Lists.
> We ask that you be polite and do the same.  Disregard for
the list etiquette
> may cause your account to be moderated.
>
> _______________________________________________
> You may change your MinGW Account Options or unsubscribe
at:
> https://lists.sourceforge.net/lists/listinfo/mingw-users
> Also:
> mailto:mingw-users-
[hidden email]?subject=unsubscribe



------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: rsync fails with files over 2GB

Erwin Waterlander
Op 27-11-2014 1:04 マーズ schreef:
> Can't be helped. 32-bit arch limit. Did you know malloc will
> return bad_alloc if you try to allocate above 1 GByte?

Large File Support (LFS) is about accessing large files over 2 GB with a
32 bit program.
http://en.wikipedia.org/wiki/Large_file_support

LFS also makes accessing files on Windows network drives reliable. I
have had in the past bug reports for dos2unix on Windows network file
access. As well on Windows, as Linux binaries accessing cifs mounted
Windows drives. In both cases LFS was the solution.

Currently mingw doesn't support LFS. mingw-64 does.

regards,

Erwin


>
> [hidden email] wrote:
>
>>> Not a compiler problem. A 32-bit gcc can use 64-bit
> integers.
>>> The problem is that MSYS defines off_t as a 32-bit integer
> [1].
>>> Because off_t has a sign bit, the maximum file size is 2GB
> - 1.
>>> There is no way to fix this without breaking compatibility
> in MSYS:
>>> one who changes off_t to 64-bit would also need to
> recompile all
>>> programs that use MSYS.
>>> I can rsync a 2.2G file with MSYS2
>>> <http://sourceforge.net/projects/msys2/>, so MSYS2 must
> have fixed
>>> this.
>>> --George Koehler
>>>
>>> [1] See 'typedef long off_t;' in
>>> http://sourceforge.net/p/mingw/msys-
> runtime/ci/master/tree/newlib/libc/include/sys/types.h
>> It has been a while since this thread was started.  Has
> there been any
>> effort to improve large file support in MSYS?  Also, I took
> a look at
>> MSYS2 and I couldn't find rsync.exe.  George, was there an
> additional
>> package you installed to get that binary?
>>
>>
>>
> ------------------------------------------------------------------------------
>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT
> Server
>> from Actuate! Instantly Supercharge Your Business Reports
> and Dashboards
>> with Interactivity, Sharing, Native Excel Exports, App
> Integration & more
>> Get technology previously reserved for billion-dollar
> corporations, FREE
> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
>> _______________________________________________
>> MinGW-users mailing list
>> MinGW-
> [hidden email]
>> This list observes the Etiquette found at
>> http://www.mingw.org/Mailing_Lists.
>> We ask that you be polite and do the same.  Disregard for
> the list etiquette
>> may cause your account to be moderated.
>>
>> _______________________________________________
>> You may change your MinGW Account Options or unsubscribe
> at:
>> https://lists.sourceforge.net/lists/listinfo/mingw-users
>> Also:
>> mailto:mingw-users-
> [hidden email]?subject=unsubscribe
>
>

--
Erwin Waterlander
http://waterlan.home.xs4all.nl/


------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: rsync fails with files over 2GB

Renato Silva
In reply to this post by Dan Raymond

2014-11-26 17:21 GMT-02:00 <[hidden email]>:

It has been a while since this thread was started.  Has there been any
effort to improve large file support in MSYS?  Also, I took a look at
MSYS2 and I couldn't find rsync.exe.

pacman -S rsync


------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe