Viewing Issue Advanced Details Jump to Notes ] Issue History ] Print ]
ID Category Severity Reproducibility Date Submitted Last Update
0003606 [Talend Open Studio] Perl components feature N/A 2008-04-18 22:38 2008-05-14 22:06
Reporter plegall View Status public  
Assigned To
Priority normal Resolution fixed Platform
Status closed   OS
Projection none   OS Version
ETA none Fixed in Version 2.4.0RC1 Product Version
  Target Version 2.4.0 Product Build
Summary 0003606: parallel executions on iterate link
Description We would like to parallelize execution after an iterate link. For example, a tFileList finds 100 files and process each of them after the iterate link. It would be useful to set a maximum of 4 parallel executions.

On the designer side, we need an option on the iterate link "enable parallel execution" and "number of parallel executions".
Steps To Reproduce
Additional Information
Tags No tags attached.
generated language
workload (in half days)
Attached Files

- Relationships
related to 0003694closedmhirt parallel executions on iterate link 

-  Notes
(0008245)
plegall (reporter)
2008-04-18 22:48

For code generation, added on trunk in [Subversion, tos] r13388

nrousseau, can you add the feature on the designer side? Once graphically added, can you modify:

- org.talend.designer.codegen/resources/component_part_footer.perljet
- org.talend.designer.codegen/resources/component_part_header.perljet

in order to use the new iterate link variable instead of :

boolean parallelIterate = false;
int nbParallelIterate = 2;

(which were commited so that you have something to test on)

For your tests, I propose you to design:

tFileList --iterate--> tFID --row--> tFOD

tFID file : $_globals{tFileList_1}{CURRENT_FILEPATH}
tFOD file : '/tmp/'.$_globals{tFileList_1}{CURRENT_FILE}.'.out'

and make sure you have several files matching the tFileList pattern.
User avatar (0008326)
nrousseau (administrator)
2008-04-23 09:02

added on trunk:

[Subversion, tos] r13493


Note that this have been added only for Perl, it can be added for Java if necessary once Java can support also this option.
(0008349)
plegall (reporter)
2008-04-23 19:16

It's nice *but*:

1. I would like to see the number of parallel executions under the iterate link in the designer

2. I would like to have a text box instead of listbox for the number of the parallel executions (so that I can use a context variable)
(0008372)
plegall (reporter)
2008-04-25 00:40

nrousseau, you have added things related to this feature in [Subversion, tos] r13508

I don't know where the problem comes from, but I haven't been able to use a context variable in the new text field.

I still can't see the "x4" in the designer :-/
User avatar (0008389)
nrousseau (administrator)
2008-04-25 11:45

modified the perljet and the display of the Iterate link.

[Subversion, tos] r13554
(0008399)
plegall (reporter)
2008-04-25 15:51

Thank you nrousseau, it's ok now.

- Issue History
Date Modified Username Field Change
2008-04-18 22:38 plegall New Issue
2008-04-18 22:38 plegall Assigned To => plegall
2008-04-18 22:48 plegall Note Added: 0008245
2008-04-18 22:48 plegall Assigned To plegall => nrousseau
2008-04-18 22:48 plegall Status new => assigned
2008-04-21 04:39 nrousseau Status assigned => feedback from Talend
2008-04-23 09:02 nrousseau Note Added: 0008326
2008-04-23 09:02 nrousseau Assigned To nrousseau => plegall
2008-04-23 09:02 nrousseau Status feedback from Talend => toVerify
2008-04-23 19:16 plegall Note Added: 0008349
2008-04-25 00:40 plegall Note Added: 0008372
2008-04-25 11:45 nrousseau Note Added: 0008389
2008-04-25 15:51 plegall Note Added: 0008399
2008-04-25 15:55 plegall Status toVerify => resolved/needs doc
2008-04-25 15:55 plegall Fixed in Version => 2.4.0RC1
2008-04-25 15:55 plegall Resolution open => fixed
2008-04-29 02:01 mhirt Issue cloned 0003694
2008-04-29 02:01 mhirt Relationship added related to 0003694
2008-05-14 22:06 plegall Status resolved/needs doc => closed


Mantis 1.1.6[^]
Copyright © 2000 - 2007 Mantis Group
Powered by Mantis Bugtracker
Copyright © 2006 - 2010 Talend. All rights reserved. Talend Contributor Agreement