Talend Exchange is the place where Talend community can share items related to Talend opensource products, such as Data Integration, Data Quality and Data Master Management. Contribution is open to any user, no specific validation is needed. As soon as you have your forum account, you automatically get a Talend Exchange account.


tGoogleAnalyticsInput


  • Author: jlolling
  • Categories: Component
  • First revision date: 2012-06-03
  • Latest revision date: 2014-02-27
  • Compatible with: Data Integration releases 4.1.3, 4.2.0, 4.2.0M1, 4.2.0M2, 4.2.0M3, 4.2.0M4, 4.2.0RC1, 4.2.0RC2, 4.2.0RC3, 4.2.1, 4.2.2, 4.2.3, 4.2.4, 5.0.0, 5.0.0M1, 5.0.0M2, 5.0.0M3, 5.0.0M4, 5.0.0M5, 5.0.0RC1, 5.0.0RC2, 5.0.0RC3, 5.0.1, 5.0.2, 5.1.0, 5.1.0M1, 5.1.0M2, 5.1.0RC1, 5.1.1, 5.1.2, 5.1.3, 5.2.0, 5.2.0M1, 5.2.0M2, 5.2.0RC1, 5.2.0RC2, 5.2.1, 5.2.2, 5.2.3, 5.3.0, 5.3.0M1, 5.3.0M2, 5.3.0M3, 5.3.0RC2, 5.3.1, 5.3.2, 5.4.0, 5.4.0M1, 5.4.0M2, 5.4.0M3, 5.4.0RC1, 5.4.1
  • Downloads: 1447

About: Lets you query Google Analytics data.
You find this component in the palette in section Business->Google.
This component use the latest GA-API version 3 (r81 v1.17.0 rc) and OAuth-API 2.0.
A configured service account is needed and must be added to your Analytics Account (+profile).
Component returns (after):
- error_message,
- amount or rows used to create result set
- flag if data are sampled
- number of lines delivered
The dimensions and metrics are defined in the notation of the Google API.
In the Advance settings you can optimize the performance by changing the fetch size and avoid problems by changing the timeout (also used for read timeout).
There is also an advanced option to reuse the client (avoid multiple logins in iterations).
Contact: jan.lolling@gmail.com
Please read the help page linked in the component detail page.
In case of invalid grant errors: please check the system time on your machine and the API grants in the Google API Console
In case of permission denied errors, please check if you have added the service account to your profiles.

*NEW*: Sampling level can be set and in case of sampling, the size and space will returned (as return values --> Outline view)
Thanks to Hans Ressing for sharing ideas and test support!
*NEW*: Has the Option Die On Error (default = true)

Related links

Revision list

expand/collapse all

Revision 1.14 97 Downloads, Released on 2014-02-27
Download revision 1.14

Compatible with: 5.4.1, 5.4.0, 5.3.2, 5.3.1, 5.3.0, 5.2.3, 5.2.2, 5.2.1, 5.2.0, 5.1.3, 5.1.2, 5.1.1

Option added Die On Error (default = true to avoid breaking the current behavior)

Revision 1.13 53 Downloads, Released on 2014-02-09
Download revision 1.13

Compatible with: 5.4.1, 5.4.0, 5.3.2, 5.3.1, 5.3.0, 5.2.3, 5.2.2, 5.2.1, 5.2.0, 5.1.3, 5.1.2, 5.1.1, 5.1.0, 5.0.2, 5.0.1, 5.0.0

Updated GA library and
enable to set the sampling level and added return values for sampling rate and sampling size

Revision 1.12 312 Downloads, Released on 2013-08-02
Download revision 1.12

Compatible with: 5.4.1, 5.4.0, 5.4.0RC1, 5.4.0M3, 5.4.0M2, 5.4.0M1, 5.3.1, 5.3.0, 5.2.3, 5.2.2, 5.2.1, 5.2.0, 5.1.3, 5.1.2, 5.1.1, 5.1.0, 5.0.2, 5.0.1, 5.0.0, 4.2.4, 4.2.3, 4.2.2, 4.2.1

Updated Google Core Reporting API 1.15 rc used.

Revision 1.11 286 Downloads, Released on 2013-02-21
Download revision 1.11

Compatible with: 5.4.0M1, 5.3.1, 5.3.0, 5.3.0RC2, 5.3.0M3, 5.3.0M2, 5.3.0M1, 5.2.2, 5.2.1, 5.2.0, 5.2.0RC2, 5.2.0RC1, 5.2.0M2, 5.2.0M1, 5.1.3, 5.1.2, 5.1.1, 5.1.0, 5.1.0RC1, 5.1.0M2, 5.1.0M1, 5.0.2, 5.0.1, 5.0.0, 5.0.0RC3, 5.0.0RC2, 5.0.0RC1, 5.0.0M5, 5.0.0M4, 5.0.0M3, 5.0.0M2, 5.0.0M1, 4.2.4, 4.2.3

Advanced Option to limit the maximum number of delivered rows. Keep the attribute empty or set it to 0 to avoid limitation.

Revision 1.10 75 Downloads, Released on 2013-01-11
Download revision 1.10

Compatible with: 5.2.1, 5.2.0, 5.2.0RC2, 5.2.0RC1, 5.2.0M2, 5.2.0M1, 5.1.3, 5.1.2, 5.1.1, 5.1.0, 5.1.0RC1, 5.1.0M2, 5.1.0M1, 5.0.2, 5.0.1, 5.0.0, 5.0.0RC3, 5.0.0RC2, 5.0.0RC1, 5.0.0M5, 5.0.0M4, 5.0.0M3, 5.0.0M2, 5.0.0M1, 4.2.4, 4.2.3

New release of Google Reporting API v3-rev24-1.12.0-beta used.

Revision 1.9 152 Downloads, Released on 2012-09-12
Download revision 1.9

Compatible with: 5.2.1, 5.2.0, 5.2.0RC2, 5.2.0M2, 5.2.0M1, 5.1.1, 5.1.0, 5.1.0RC1, 5.1.0M2, 5.1.0M1, 5.0.2, 5.0.1, 5.0.0, 5.0.0RC3, 5.0.0RC2, 5.0.0RC1, 5.0.0M5, 5.0.0M4, 5.0.0M3, 5.0.0M2, 5.0.0M1, 4.2.4, 4.2.3, 4.2.2, 4.2.1

Code generation optimization.
Avoid problems with not null values.

If you expierence problems with missing jars, please delete the file configuration/ComponentCache.javacache and restart Studio. (If you use the commandline do the same for it!).
If that does not help - please keep in mind these are all problems caused by Talend not by the component - do following:
1. copy the component to any directory (and keep this dir in mind) of your computer
2. In Preferences set Talend Component Development directory to the former choosed directory where you copied the component to
3. Switch to the component development perspective
4. right klick with the mouse on component dir and choose Push components to palette. After doing that, the Generation Engine should restart automatically.

That cause Talend to intergrate components in the strongest way.

Revision 1.8 44 Downloads, Released on 2012-09-05
Download revision 1.8

Compatible with: 5.2.0M2, 5.2.0M1, 5.1.1, 5.1.0, 5.1.0RC1, 5.1.0M2, 5.0.0RC3, 5.0.0RC2, 5.0.0RC1, 5.0.0M5, 5.0.0M4, 5.0.0M3, 5.0.0M2, 5.0.0M1, 4.2.4, 4.2.3, 4.2.2, 4.2.1, 4.2.0, 4.2.0RC3, 4.2.0RC2, 4.2.0RC1, 4.2.0M4

New API version r15 1.8.0 used. This is necessary to avoid conflicts with the new component tGoogleAnalyticsMCFInput which depends on this version.
There is a problem with the code generator if this component has no out going connections attached. In this case the code generator fails, because of this component need an out going connection it shouldn t be a problem.

If you expierence problems with missing jars, please delete the file configuration/ComponentCache.javacache and restart Studio. (If you use the commandline do the same for it!).
If that does not help - please keep in mind these are all problems caused by Talend not by the component - do following:
1.copy thes component to any location of your computer
2. In Preferences set Talend Component Development directory to the directory where you copied the component
3. Switch to the component development perspective
4. klick with mous on component and choose Push components to palette.

That cause Talend to intergrate components in the strongest way.

Revision 1.7 63 Downloads, Released on 2012-08-18
Download revision 1.7

Compatible with: 5.1.1, 5.1.0, 5.1.0RC1, 5.1.0M2, 5.1.0M1, 5.0.2, 5.0.1, 5.0.0, 5.0.0RC3, 5.0.0RC2, 5.0.0RC1, 5.0.0M5, 5.0.0M4, 5.0.0M3, 5.0.0M2, 5.0.0M1, 4.2.4, 4.2.3, 4.2.2, 4.2.1

Problem solved with dimension ga:date. Please ensure, that schema has date pattern yyyyMMdd (double qouted) for ga:date dimension.
The locale for number format is now requiered. Fortunately the default value is set correctly, so there is no need to change that in 99,99% of all cases.

Revision 1.6 68 Downloads, Released on 2012-07-31
Download revision 1.6

Compatible with: 5.1.1, 5.1.0, 5.1.0RC1, 5.1.0M2, 5.1.0M1, 5.0.2, 5.0.1, 5.0.0RC3, 5.0.0RC2, 5.0.0RC1, 5.0.0M5, 5.0.0M4, 5.0.0M3, 5.0.0M2, 5.0.0M1, 4.2.4, 4.2.3, 4.2.2, 4.2.1, 4.2.0, 4.2.0RC3, 4.2.0RC2, 4.2.0RC1, 4.2.0M4, 4.2.0M3, 4.2.0M2, 4.2.0M1, 4.1.3

French translation added (special thanks to Nicolas Marchand and my wife Doris)

Revision 1.5 110 Downloads, Released on 2012-06-20
Download revision 1.5

Compatible with: 5.1.1, 5.1.0, 5.1.0RC1, 5.1.0M2, 5.1.0M1, 5.0.2, 5.0.1, 5.0.0, 5.0.0RC3, 5.0.0RC2, 5.0.0RC1, 5.0.0M5, 5.0.0M4, 5.0.0M3, 5.0.0M2, 5.0.0M1, 4.2.4, 4.2.3, 4.2.2

* New library version google-analytics-lib used (v1.7.1 beta)
* Problem with \"invalid grant\" (or in former version \"login required\") error solved:
This problem happens if the application runs in the future from Google point of view, means if the local clock is a bit to early.
To avoid that problem, there is a new advanced option to add a time offset into the past to be sure that the component runs always in the past from googles point of view, otherwise refreshing access token fails.
* Dimensions are not requiered anymore.

Revision 1.4 42 Downloads, Released on 2012-06-14
Download revision 1.4

Compatible with: 5.1.1, 5.1.0, 5.1.0RC1, 5.1.0M2, 5.1.0M1, 5.0.2, 5.0.1, 5.0.0, 5.0.0RC3, 5.0.0RC2, 5.0.0RC1, 5.0.0M5, 5.0.0M4, 5.0.0M3, 5.0.0M2, 5.0.0M1, 4.2.4, 4.2.3, 4.2.2

Bugs fixed:
If running in iteration with reusage client, start index will not start index at 1
NullPointerException if last paging returns zero data sets.

Revision 1.3 39 Downloads, Released on 2012-06-14
Download revision 1.3

Compatible with: 5.1.1, 5.1.0, 5.1.0RC1, 5.1.0M2, 5.1.0M1, 5.0.2, 5.0.1, 5.0.0, 5.0.0RC3, 5.0.0RC2, 5.0.0RC1, 5.0.0M5, 5.0.0M4, 5.0.0M3, 5.0.0M2, 5.0.0M1, 4.2.4, 4.2.3, 4.2.2

Fixed client reusage.
Improved client sharing with key extension.
The Key to reuse the client will build by:
component id + jobName + optinal given additional key (to separate the clients if you plan to run the same job simultaniously)

Revision 1.2 53 Downloads, Released on 2012-06-08
Download revision 1.2

Compatible with: 5.1.1, 5.1.0, 5.1.0RC1, 5.1.0M2, 5.1.0M1, 5.0.2, 5.0.1, 5.0.0, 5.0.0RC3, 5.0.0RC2, 5.0.0RC1, 5.0.0M5, 5.0.0M4, 5.0.0M3, 5.0.0M2, 5.0.0M1, 4.2.4, 4.2.3, 4.2.2, 4.2.1

adds more return values,
improved performance,
totals can included into the resultset
Please be aware, that the componente job properties will be reset if you use an older version already in your jobs!

Revision 1.0 53 Downloads, Released on 2012-06-03
Download revision 1.0

Compatible with: 5.1.1, 5.1.0, 5.1.0RC1, 5.1.0M2, 5.1.0M1, 5.0.2, 5.0.1, 5.0.0, 5.0.0RC3, 5.0.0RC2, 5.0.0RC1, 5.0.0M5, 5.0.0M4, 5.0.0M3, 5.0.0M2, 5.0.0M1, 4.2.4, 4.2.3, 4.2.2, 4.2.1

first tested version

Reviews (10)

 Great Component By amenezes on April 14, 2014
Fantastic work!
 Auth problem By MedAchraf on March 5, 2014
when trying to connect I get this error:

Exception in component tGoogleAnalyticsInput_1
com.google.gdata.util.AuthenticationException: Error connecting with login URI
at com.google.gdata.client.GoogleAuthTokenFactory.getAuthToken(GoogleAuthTokenFactory.java:489)
at com.google.gdata.client.GoogleAuthTokenFactory.setUserCredentials(GoogleAuthTokenFactory.java:346)
at com.google.gdata.client.GoogleService.setUserCredentials(GoogleService.java:362)
at com.google.gdata.client.GoogleService.setUserCredentials(GoogleService.java:317)
at com.google.gdata.client.GoogleService.setUserCredentials(GoogleService.java:301)
at googleanalyticstes.googletest_0_1.googleTest.tGoogleAnalyticsInput_1Process(googleTest.java:574)
at googleanalyticstes.googletest_0_1.googleTest.runJobInTOS(googleTest.java:1089)
at googleanalyticstes.googletest_0_1.googleTest.main(googleTest.java:947)
Caused by: java.net.ConnectException: Connection timed out: connect
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(Unknown Source)
at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.connect(Unknown Source)
at com.sun.net.ssl.internal.ssl.BaseSSLSocketImpl.connect(Unknown Source)
at sun.net.NetworkClient.doConnect(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.protocol.https.HttpsClient.(Unknown Source)
at sun.net.www.protocol.https.HttpsClient.New(Unknown Source)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(Unknown Source)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(Unknown Source)
at com.google.gdata.client.GoogleAuthTokenFactory.makePostRequest(GoogleAuthTokenFactory.java:551)
at com.google.gdata.client.GoogleAuthTokenFactory.getAuthToken(GoogleAuthTokenFactory.java:487)
... 7 more



I think this may be related to the advanced setting parameters
Hi,

Can anyone tell me that what am i missing? Thanks.

http://www.talendforge.org/forum/viewtopic.php?id=29714

regards,
 dependency still not found By mlindman on March 6, 2013
I tried the steps for problemsolving, but I'm still getting the error "Module cimt.talendcomp.gaanalytics-1.5.0-jar-with-dependencies.jar required"

Is there something that would prevent this module from working on Enterprise version (with svn)?

Are there any steps I could take to fix this manually?
 Directeur Informatique By Pierret on February 13, 2013
One functionality missing.

This is a great component and it works fine but Iit's not possible to limit the results. I don't want to store all the results in my database. Is it possible to add a parameter to set max-results?

Thanks
 Can't make it work with Talend 5.1.1 By atienza on February 13, 2013
I just downloaded the last version but when trying to launch the job, I get errors (in the code windows) indicating this :
de.cimt.talendcomp.googleanalytics.GoogleAnalyticsInput
"de. cannot be resolved"
Did I miss anything in the installation process ?
 Great component By BZIKHA Rabi on January 15, 2013
I already use this component.It's very good and it is important in Datamanagement.

Tanks,
 Great Tool By ketan_dangi on September 21, 2012
Other then initial installation problem its really great tool.
 Jar not found Issue By ketan_dangi on September 11, 2012
After downloading above latest version and two steps still getting "...1.4-dependency.jar not found".


After fresh install its works fine. By ketan_dangi on September 21, 2012
After installing again above error no longer exist. now I can say this is really great tool
 Great component! By mkeyes on July 18, 2012
Works like a charm, and since it uses OAuth it is about as future proof as you can get. Two suggestions from the author to get it to work right out of the box:

1) After you create your service account key in the Google API Console add your service account email as a user of the GA profiles you're using.
2) Delete the ComponentCache.javacache in the configuration subdirectory and restart Studio to get rid of the "jar not found" error after install.

After I did these two steps it worked perfectly.


Exception in component tGoogleAnalyticsInput_1 com.google.api.client.auth.oauth2.TokenResponseException: 400 Bad Request { \"error\" : \"invalid_grant\" } at com.google.api.client.auth.oauth2.TokenResponseException.from(TokenResponseException.java:103) at com.google.api.client.auth.oauth2.TokenRequest.executeUnparsed(TokenRequest.java:303) at com.google.api.client.auth.oauth2.TokenRequest.execute(TokenRequest.java:323) at com.google.api.client.googleapis.auth.oauth2.GoogleCredential.executeRefreshToken(GoogleCredential.java:340) at com.google.api.client.auth.oauth2.Credential.refreshToken(Credential.java:505) at com.google.api.client.auth.oauth2.Credential.intercept(Credential.java:266) at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:857) at com.google.api.client.googleapis.json.GoogleJsonResponseException.execute(GoogleJsonResponseException.java:182) at com.google.api.client.googleapis.services.GoogleClient.executeUnparsed(GoogleClient.java:279) at com.google.api.client.http.json.JsonHttpRequest.executeUnparsed(JsonHttpRequest.java:207) at com.google.api.services.analytics.Analytics$Data$Ga$Get.execute(Analytics.java:1622) at de.cimt.talendcomp.googleanalytics.GoogleAnalyticsInput.executeDataQuery(GoogleAnalyticsInput.java:260) at de.cimt.talendcomp.googleanalytics.GoogleAnalyticsInput.executeQuery(GoogleAnalyticsInput.java:303) at dheeraj.gtest_0_1.gtest.tGoogleAnalyticsInput_1Process(gtest.java:493) at dheeraj.gtest_0_1.gtest.runJobInTOS(gtest.java:822) at dheeraj.gtest_0_1.gtest.main(gtest.java:690) [statistics] disconnected
Submit review
Name:*
Email:*
Title:*
Please select your rating*
Review:*


Version Author Released on Rating Downloads
ParserRule

Tweets

1.0 scorreia 2013-11-20
18

get information from tweets.
Extract the date/time, user, hashtags, referenced users and urls from Twitter messages.

Regex

Only alphabetical characters not empty

1.0 dcortinovis 2013-06-19
62

Only alphabetical characters not empty.
And at least one (empty forbidden)

Indicator

EMail validation via mail server

5.4/5.3 mzhao 2013-06-03
517

This Java UDI check emails by sending a SMTP request to mail server. the code sample can be found at: http://www.rgagnon.com/javadetails/java-0452.html

Indicator

Frequency table of hours

2.0 scorreia 2013-04-25
355

This indicator helps to analyze the most frequent day hours that appear in date time columns.

Indicator

Sample Standard Deviation

1.1 scorreia 2013-04-25
268

This indicator computes the sample standard deviation of any numerical column

Indicator

Variance

1.1 scorreia 2013-04-25
249

This indicator computes the variance of numeric columns

Indicator

Trimmed

1.0 scorreia 2013-04-25
60

evaluate the number of data which are correctly trimmed

Indicator

Week Frequency

2.0 scorreia 2013-04-25
270

aggregates Date fields into weeks

Indicator

Duplicate Rows

2.0 scorreia 2013-04-25
774

this indicator counts the number of duplicate rows.
It's different from the system indicator called "duplicate count" because it counts the number of duplicate rows, not the number of duplicate values.

Indicator

Length Range Frequency

1.1 scorreia 2013-04-25
122

get length ranges of data.

group data according to their length range.
Ranges are the following:
data of length < 10
data of length < 20
data of length < 30
data of length >= 30
null data

Version Author Released on Rating Downloads
Export

Product Demo

3.0 ctoum 2012-05-31
559

Product & families, with Cafepress pictures.

Data-Model

Clinical Trials: Janus Model Basics

1.0 jaymce 2010-11-22
376

This is a model of the basic of the Janus Clinical Data Repository.
http://www.fda.gov/ForIndustry/DataStandards/StudyDataStandards/ucm155327.htm

Data-Model

D* Demo Model

1.0 ctoum 2010-08-13
705

Model used in the D* Demo.

Export

Talendshop Demo

1.0 ctoum 2010-08-04
1109

Talendshop Demo (Demo Project)


69 ms