Generating 2.5 mill UIDs and saving them to a file

Hi

I need around 2.5 mill UIDs for use as event UIDs, but the following only returns 10,000 (this might be a browser setting - not sure):

http://localhost:8080/dhis//api/26/system/id.csv?limit=2500000&paging=false

Any other way to generate those 2.5 mill UIDs and dump them directly out into a csv file?

Regards

calle

···

Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg


By courtesy of Bob:

https://github.com/dhis2/dhis2-utils/blob/master/resources/sql/uid.sql

···

On Sun, Jun 25, 2017 at 11:33 PM, Calle Hedberg calle.hedberg@gmail.com wrote:

Hi

I need around 2.5 mill UIDs for use as event UIDs, but the following only returns 10,000 (this might be a browser setting - not sure):

http://localhost:8080/dhis//api/26/system/id.csv?limit=2500000&paging=false

Any other way to generate those 2.5 mill UIDs and dump them directly out into a csv file?

Regards

calle


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg



Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Knut Staring

Dept. of Informatics, University of Oslo

Norway: +4791880522

Skype: knutstar

http://dhis2.org

Knut

Aware of that, but it generates only one UID. I can use it in a small program, of course, but that’s not the point here:

I would like to know if there is a way to get the web API to create 2.5 mill UIDs into a csv file.

Regards

Calle

···

On 25 June 2017 at 23:45, Knut Staring knutst@gmail.com wrote:

By courtesy of Bob:

https://github.com/dhis2/dhis2-utils/blob/master/resources/sql/uid.sql

On Sun, Jun 25, 2017 at 11:33 PM, Calle Hedberg calle.hedberg@gmail.com wrote:

Hi

I need around 2.5 mill UIDs for use as event UIDs, but the following only returns 10,000 (this might be a browser setting - not sure):

http://localhost:8080/dhis//api/26/system/id.csv?limit=2500000&paging=false

Any other way to generate those 2.5 mill UIDs and dump them directly out into a csv file?

Regards

calle


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg



Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Knut Staring

Dept. of Informatics, University of Oslo

Norway: +4791880522

Skype: knutstar

http://dhis2.org


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg


Call the same end point 250 times maybe and concatenate to a file? :confused:

···

On Jun 25, 2017 18:11, “Calle Hedberg” calle.hedberg@gmail.com wrote:

Knut

Aware of that, but it generates only one UID. I can use it in a small program, of course, but that’s not the point here:

I would like to know if there is a way to get the web API to create 2.5 mill UIDs into a csv file.

Regards

Calle


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

On 25 June 2017 at 23:45, Knut Staring knutst@gmail.com wrote:

By courtesy of Bob:

https://github.com/dhis2/dhis2-utils/blob/master/resources/sql/uid.sql


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg


On Sun, Jun 25, 2017 at 11:33 PM, Calle Hedberg calle.hedberg@gmail.com wrote:

Hi

I need around 2.5 mill UIDs for use as event UIDs, but the following only returns 10,000 (this might be a browser setting - not sure):

http://localhost:8080/dhis//api/26/system/id.csv?limit=2500000&paging=false

Any other way to generate those 2.5 mill UIDs and dump them directly out into a csv file?

Regards

calle


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg



Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Knut Staring

Dept. of Informatics, University of Oslo

Norway: +4791880522

Skype: knutstar

http://dhis2.org

Hi Calle

Just use something like this (you need to have the uid() function installed first)

copy (select uid() as uid from generate_series(1,10)) to ‘/tmp/test.csv’;

and replace 10 with whatever number you want

···

On Mon, Jun 26, 2017 at 7:25 AM, Jason Pickering jason.p.pickering@gmail.com wrote:

Call the same end point 250 times maybe and concatenate to a file? :confused:


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Jun 25, 2017 18:11, “Calle Hedberg” calle.hedberg@gmail.com wrote:

Knut

Aware of that, but it generates only one UID. I can use it in a small program, of course, but that’s not the point here:

I would like to know if there is a way to get the web API to create 2.5 mill UIDs into a csv file.

Regards

Calle


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

On 25 June 2017 at 23:45, Knut Staring knutst@gmail.com wrote:

By courtesy of Bob:

https://github.com/dhis2/dhis2-utils/blob/master/resources/sql/uid.sql


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg


On Sun, Jun 25, 2017 at 11:33 PM, Calle Hedberg calle.hedberg@gmail.com wrote:

Hi

I need around 2.5 mill UIDs for use as event UIDs, but the following only returns 10,000 (this might be a browser setting - not sure):

http://localhost:8080/dhis//api/26/system/id.csv?limit=2500000&paging=false

Any other way to generate those 2.5 mill UIDs and dump them directly out into a csv file?

Regards

calle


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg



Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Knut Staring

Dept. of Informatics, University of Oslo

Norway: +4791880522

Skype: knutstar

http://dhis2.org

(and no, there is a limit for generating UIDs in the web-api for a reason… that endpoint is open to everyone, and you could kill servers if we allowed no upper limit)

···

On Mon, Jun 26, 2017 at 8:16 AM, Morten Olav Hansen morten@dhis2.org wrote:

Hi Calle

Just use something like this (you need to have the uid() function installed first)

copy (select uid() as uid from generate_series(1,10)) to ‘/tmp/test.csv’;

and replace 10 with whatever number you want

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Mon, Jun 26, 2017 at 7:25 AM, Jason Pickering jason.p.pickering@gmail.com wrote:

Call the same end point 250 times maybe and concatenate to a file? :confused:


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

On Jun 25, 2017 18:11, “Calle Hedberg” calle.hedberg@gmail.com wrote:

Knut

Aware of that, but it generates only one UID. I can use it in a small program, of course, but that’s not the point here:

I would like to know if there is a way to get the web API to create 2.5 mill UIDs into a csv file.

Regards

Calle


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

On 25 June 2017 at 23:45, Knut Staring knutst@gmail.com wrote:

By courtesy of Bob:

https://github.com/dhis2/dhis2-utils/blob/master/resources/sql/uid.sql


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg


On Sun, Jun 25, 2017 at 11:33 PM, Calle Hedberg calle.hedberg@gmail.com wrote:

Hi

I need around 2.5 mill UIDs for use as event UIDs, but the following only returns 10,000 (this might be a browser setting - not sure):

http://localhost:8080/dhis//api/26/system/id.csv?limit=2500000&paging=false

Any other way to generate those 2.5 mill UIDs and dump them directly out into a csv file?

Regards

calle


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg



Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Knut Staring

Dept. of Informatics, University of Oslo

Norway: +4791880522

Skype: knutstar

http://dhis2.org

Morten,

Firstly, thanks for confirming there’s a hard-coded limit - I suspected it, but found no reference to any limits in the developer manual.

Secondly, I did generate them easily enough using Bob’s uid function and a small temp table to store the uids:

DO

$do$

BEGIN

FOR i IN 1…2411000 LOOP

INSERT INTO aa_uid SELECT uid() as uidno;

END LOOP;

END

$do$;

BUT your use of COPY and GENERATE_SERIES is neater with no need for a temp table - thanks for that! Works like a charm…

Regards

Calle

···

On 26 June 2017 at 03:18, Morten Olav Hansen morten@dhis2.org wrote:

(and no, there is a limit for generating UIDs in the web-api for a reason… that endpoint is open to everyone, and you could kill servers if we allowed no upper limit)

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Mon, Jun 26, 2017 at 8:16 AM, Morten Olav Hansen morten@dhis2.org wrote:

Hi Calle

Just use something like this (you need to have the uid() function installed first)

copy (select uid() as uid from generate_series(1,10)) to ‘/tmp/test.csv’;

and replace 10 with whatever number you want

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Mon, Jun 26, 2017 at 7:25 AM, Jason Pickering jason.p.pickering@gmail.com wrote:

Call the same end point 250 times maybe and concatenate to a file? :confused:


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

On Jun 25, 2017 18:11, “Calle Hedberg” calle.hedberg@gmail.com wrote:

Knut

Aware of that, but it generates only one UID. I can use it in a small program, of course, but that’s not the point here:

I would like to know if there is a way to get the web API to create 2.5 mill UIDs into a csv file.

Regards

Calle


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

On 25 June 2017 at 23:45, Knut Staring knutst@gmail.com wrote:

By courtesy of Bob:

https://github.com/dhis2/dhis2-utils/blob/master/resources/sql/uid.sql


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg


On Sun, Jun 25, 2017 at 11:33 PM, Calle Hedberg calle.hedberg@gmail.com wrote:

Hi

I need around 2.5 mill UIDs for use as event UIDs, but the following only returns 10,000 (this might be a browser setting - not sure):

http://localhost:8080/dhis//api/26/system/id.csv?limit=2500000&paging=false

Any other way to generate those 2.5 mill UIDs and dump them directly out into a csv file?

Regards

calle


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg



Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Knut Staring

Dept. of Informatics, University of Oslo

Norway: +4791880522

Skype: knutstar

http://dhis2.org


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg