Talend Exchange is the place where Talend community can share items related to Talend opensource products, such as Data Integration, Data Quality and Data Master Management. Contribution is open to any user, no specific validation is needed. As soon as you have your forum account, you automatically get a Talend Exchange account.


Show

Category
Search
Version
Author
 

Statistics

  • 602 extensions
  • 907 revisions
  • 294 contributors
  • 155528 downloads
 

Top Contributors

Version Author Released on Rating Downloads
Component

tMemoryMonitor

0.1 tal00000 2014-08-22
58

Monitor the memory usage of your running Job, with tMemoryMonitor

Information on the used, total and maximum (see -Xmx) memory of the Java runtime, are written to standard output or, if a warning threshold is reached, to standard error.

Options include \'Enabled\', \'Interval\' (milliseconds) and \'Warning Threshold Percent %\'.

Used in conjunction with your own diagnostic messages, this component will help you to identify where memory is being used within your Job an help you to avoid Out of Memory (OOM) Exceptions.

As well as ensuring that your Job has enough memory, you\'ll also want to make sure that you do not waste system memory. The Talend Job defaults are -Xms256MB and -Xmx1024MB. As well as some of your Jobs requiring more that 1024MB, some will require less that 256MB. This utility will allow you to observe the peak memory usage and to tune accordingly.

For more information on Java Runtime memory reporting, read http://docs.oracle.com/javase/7/docs/api/java/lang/Runtime.html.

Used memory is totalMemory - freeMemory. Percentages are based on used memory vs. maxMemory.

This component also reports the number of available processors.

Sample output.

tMemoryMonitor_1: using 43MB/246MB (max=910MB/5%) (processors=4)
tMemoryMonitor_1: using 91MB/310MB (max=910MB/10%) (processors=4)
tMemoryMonitor_1: using 260MB/449MB (max=910MB/29%) (processors=4)
tMemoryMonitor_1: using 189MB/442MB (max=910MB/21%) (processors=4)
tMemoryMonitor_1: using 259MB/454MB (max=910MB/28%) (processors=4)
tMemoryMonitor_1: using 216MB/577MB (max=910MB/24%) (processors=4)

Component

tHTMLParse

0.1 tal00000 2014-08-22
1

This component fetches and parses an HTML document with help from the excellent Java library jsoup (jsoup.org).

A number of elements are extracted from the document together with useful information including the fetch time.

Some elements, for example, Hyperlinks are made available through the jsoup class Elements, allowing you to use the power of jsoup to process these as you wish. When I have time, I\'ll write some supplemental components that will iterate through these Elements; however, it\'s simple enough to do this yourself using a component such as tJavaFlex.

The entire document is also made available through the jsoup Document class, allowing you full control over the document.

Component

tJobInstance* component collection

3.2 jlolling 2014-08-19
23

These 4 components:
- tJobInstanceStart
- tJobInstanceEnd
- tJobDataRangeScanner
- tJobInstanceLiveCheck
are dedicated to manage job monitoring.
These components provides following features:
* Creates an job instance Id to mark the datasets for data lineage
* holds all necessary information about a job run in a table
* keeps all context variables in a table (at the job start and its end)
* keeps as much as needed counters from the job
* provides incremental loads
* provides a lot of key figures about the last job run
* keeps the error messages from the job in the status table
* enables Log4J for the job with various outputs
Please refer the linked documentation
The release 3.0 needs slightly changed tables!
Please contact me in case of questions and do not use the rating function to post questions.

Component

tTimeGenerator

1.0 Sanjay Tiwari 2014-08-12
4

Component

tOpenerpOut

1.0 popeye 2014-08-12
9

Component

tOpenerp

1.0 popeye 2014-08-12
6

Component

tInfobrightOutput

3.7 T.Sasaki 2014-08-12
5

Component

tQlikviewInput

0.1 gorotman 2014-08-07
9

Reads data from QVX (QlikView data eXchange) files and pushs rows to standard flow.
It can extracts fields properties from qvx (QvxTableHeader) and creates the talend metadata xml file (ready to import into repository as generic metadata; I\'m sorry but I didn\'t found a solution to manage a custom metadata type, I read some post, blog and so but neither works fine!).
Parameters:
date format string (used for talend metadata)
charset (typically is utf-8)
data buffer size (it\'s the buffer size loaded in memory; useful if document is very large)
text buffer size (it\'s the buffer used to store text fields when reading; insert a value larger to fit you longest text value)
debug (to print debug information).

Routine

BRules

1.6 walkerca 2014-07-30
897

A collection of Java routines for validation, formatting, and logic for use in Talend Open Studio

Software license: Apache License, Version 2.0

http://www.apache.org/licenses/LICENSE-2.0

Component

Elasticsearch Indexing

1.0.3 pklalitjha 2014-07-29
238

Elasticsearch Indexing Talend Custom Component
================================================
Often, we need full text search support in our application along with analytics. Having a component will allow indexing created/updated along with
data loaded in warehouse. So, I have created a component for indexing (create/update) for Elasticsearch. Elasticsearch, is distributed, open source,
full text engine built on top of Lucene. It is well documented, supports REST API over JSON and many other native language API for indexing and querying.

NOTE: To support add or update, at least one field in schema must be designated as key to identify rows as unique. Multiple columns can be designated as key (in case of composite primary keys).

Updated extension to fix compatibility issues with 1.x version of Elasticsearch. Removed dependent jars (lucne-core and elasticsearch), so now need to add them from Elasticsearch distribution. Also reduced download size from 10MB to 15KB.

Elasticsearch v1.2+ works with JDK 7+, so if elasticsearch version is 1.2+ then use JDK 7 u25/55/65 (recommended by ES) as this component is using elasticsearch Java API and corrosponding lucene core jar.

Added support for configuring cluster name in v1.0.3

Show

Category
Search
Version
Author
 

Statistics

  • 143 extensions
  • 207 revisions
  • 38 contributors
  • 17502 downloads
 

Top Contributors

Version Author Released on Rating Downloads
Regex

Training export

0.1 cathyc 2014-06-10
9

Training export

Regex

Minutes Seconds

2.1 mhallam 2014-05-26
150

validates minutes:seconds format

Validates 3:56, 59:59,...
Does not validate 60:59 or 59:60.

SQL

hotmail email

2.0 scorreia 2014-05-26
317

filters email from hotmail.com

Regex

Dutch Postal Code

2.0 fcweeber 2014-05-26
119

Postal Code format verfication (Netherlands)

Matches
9999AA|9999 AA
Non Matches
9999aa|9999Aa|9999 aa|9999 aA

Regex

Dutch Phone Number

2.0 fcweeber 2014-05-26
120

Phone Number format verfication (Netherlands)

Local: Dutch Phonenumber format is area code (3 or 4 digits), phone number ( 7 or 6 digits) - total length (10 digits)
International: country code 0031 and then remove zero from area code
Matches
0031121234567|+31123123456|0121234567|012-1234567|0123-123456
Non Matches
012 1234567|1234567
match 02.31.23.45.67.22 or 004923123467223

Regex

Names with unicode characters

2.0 scorreia 2014-05-26
117

Match people names with unicode characters.
Matches Jean-Marc, Jørn, Mc\'Neelan, Pz. López
Does not match I.B.M.

Regex

FR Phone Number (parenthesis allowed)

2.0 scorreia 2014-05-26
331

match French phone numbers in several format:
matches:
0033 1 47 25 00 00
+33 1 47 25 00 00
(33) 1 47 25 00 00
0033147250000
01-47-25-00-00
01 47 25 00 00
Does not match
0147 250 000

Regex

Email Address (with list of top-level domains)

2.0 mzhao 2014-05-26
181

Check the validity of email addresses.

Regex

Regular Text

2.0 scorreia 2014-05-26
96

match regular text

matches regular text such as \"hello Jean-Baptiste\".
does not match text with any special character such as \"# num\", \"test;\"

Regex

Text

2.0 scorreia 2014-05-26
74

match regular text with punctuation correctly placed.

Show

Category
Search
Version
Author
 

Statistics

  • 5 extensions
  • 7 revisions
  • 4 contributors
  • 4002 downloads
 

Top Contributors

Version Author Released on Rating Downloads
Export

Product Demo

3.0 ctoum 2012-05-31
603

Product & families, with Cafepress pictures.

Data-Model

Clinical Trials: Janus Model Basics

1.0 jaymce 2010-11-22
402

This is a model of the basic of the Janus Clinical Data Repository.
http://www.fda.gov/ForIndustry/DataStandards/StudyDataStandards/ucm155327.htm

Data-Model

D* Demo Model

1.0 ctoum 2010-08-13
739

Model used in the D* Demo.

Export

Talendshop Demo

1.0 ctoum 2010-08-04
1130

Talendshop Demo (Demo Project)


64 ms