Important Notice: Our web hosting provider recently started charging us for additional visits, which was unexpected. In response, we're seeking donations. Depending on the situation, we may explore different monetization options for our Community and Expert Contributors. It's crucial to provide more returns for their expertise and offer more Expert Validated Answers or AI Validated Answers. Learn more about our hosting issue here.

What are the limits of Rserve in terms of data size?

April 26, 2017Data limits rserve size terms

0

Posted

What are the limits of Rserve in terms of data size?

1 Answer

0

Posted

Limits in versions up to 0.2-x: • Maximum size of a single REXP: 16MB (both ways, fixed) • Default packet size: 2MB (max. 1GB) configurable via CMD_setBufSize resp. setBufferSize in JRclient Limits in version 0.3: • Maximum size of a single REXP: 2GB (on 32-bit platforms), theoretical limit is 2^55 on 64-bit platforms. • Packet size is auto-adjusted, configured by maxinbuf and maxsendbuf config entries. (maximum 2GB) The maxinbuf (max. packet from client to Rserve) and maxsendbuf (max. packet from Rserve to client) options in the configuration file allow you to set limits in order to prevent memory overflow on machines that act as servers for multiple connections. The defaults are 16MB and unlimited respectively. BTW: 16MB corresponds roughly to 2 mio real numbers, so 1000×2000 matrices is about as big as you can go in versions 0.2 and lower. A work-around is to transmit data per-partes, e.g. by column or row (or use Rserve 0.3 or higher). Data conversion on-the-fly (at least outgoing)