Connecting to Email

Concept Information

ACCESSDATA command

Use the Email data connector to import email messages for a single account using the standard mail protocols IMAP or POP. When you connect to your email server, each table name represents a mailbox folder on the server, and each record represents an email message.

Note

You cannot use this connector independently of Analytics. You can configure a DSN for the connector driver using the Windows ODBC Data Source Administrator, however you must test the DSN connection from within Analytics and not from the connector's Windows DSN configuration dialog.

This connector retrieves email messages for a single account stored on mail servers only. It does not connect to features such as chat, or to-dos that are included with some email clients.

Before you start

To connect to Email, you must gather the following:

  • the domain name or IP address of the Email server
  • the correct connection port
  • the connecting user account, including the user name and password

Note

Your email server must use either the IMAP or POP protocol.

For help gathering the connection prerequisites, contact the Email administrator in your organization. If your administrator cannot help you, you or your administrator should contact Email Support.

Create an Email connection

  1. From the Analytics main menu, select Import > Database and application.
  2. From the New Connections tab, in the ACL Connectors section, select Email.

    Tip

    You can filter the list of available connectors by entering a search string in the Filter connections box. Connectors are listed alphabetically.

  3. In the Data Connection Settings panel, enter the connection settings and at the bottom of the panel, click Save and Connect.

    You can accept the default Connection Name, or enter a new one.

The connection for Email is saved to the Existing Connections tab. In the future, you can reconnect to Email from the saved connection.

Once the connection is established, the Data Access window opens to the Staging Area and you can begin importing data. For help importing data from Email, see Working with the Data Access window.

Connection settings

Basic settings

Setting Description Example
Protocol

The type of email server to connect to:

  • IMAP
  • POP
 IMAP
User The user of the Email account used to authenticate. Together with Password, this field is used to authenticate to the email servers. recipient@example.com
Password

The password of the email account used to authenticate. Together with User, this field is used to authenticate to the email servers.

  
Server

The name or address of the mail server.

This property specifies the IP address or the domain name of the mail server. It must be set before a connection is attempted and cannot be changed once a connection is in progress.

 imap.gmail.com
Port

The port of the mail server. The default value is:

  • IMAP 143 (non-SSL) or 993 (SSL)
  • POP 110 (non-SSL) or 995 (SSL)

A valid port number (a value between 1 and 65535) is required for the connection to take place. The property must be set before a connection is attempted and cannot be changed once a connection is established.

 993

Advanced settings

Setting Description Example
Convert Datetime to GMT

Converts datetime fields to GMT time zone during import. If false, the datetime value is converted to the operating system time zone of the machine running Analytics.

 true
Email Service

The name of the email service you are using.

Optimizes the IMAP connection for the service you are working with. You may use one of the following:

  • AOL
  • Gmail
  • Outlook
  • Yahoo
  • Other

Note

Using the Email connector, you cannot retrieve the contents of the Subject field for the Task or Calendar mailboxes from an Outlook 365 account. If you are connecting to an Outlook 365 account, consider using the Exchange connector instead. For more information, see Connecting to Exchange

 Other
Include Message

Whether to include message body content and attachment data or not.

Caution

This setting affects performance and may cause your query to timeout if you are working with many records.

 false
Is HTML

Determines whether the Message Body is HTML or plain-text.

 true
Keep Alive

Determines whether to keep the connection alive across instances.

 true
List Mailboxes

Whether to list all mailboxes or just the subscribed IMAP mailboxes. IMAP Only:

  • All
  • Subscribed
 All
Limit Key Size

The maximum length of a primary key column. Setting the size to 0 will make the key length revert to the original length.

This property makes the connector override the reported length of all the primary key columns.

 255
Map to Long Varchar

Controls whether or not a column is returned as SQL_LONGVARCHAR.

Use this setting to map any column larger than the specified size so they are reported as SQL_LONGVARCHAR instead of SQL_VARCHAR.

 -1
Map To WVarchar

Controls whether or not string types map to SQL_WVARCHAR instead of SQL_VARCHAR. It is set by default.

String columns must be mapped to SQL_WVARCHAR to accommodate various international character sets, so MapToWVarchar is set to true by default. You may set it to false to use SQL_VARCHAR instead.

 true
Max Items

Maximum number of items to return.

The default value is -1. This value ensures that all items are returned.

 -1
Pseudo Columns

Indicates whether or not to include pseudo columns as columns to the table.

This setting is particularly helpful in Entity Framework, which does not allow you to set a value for a pseudo column unless it is a table column.

The value of this connection setting is of the format "Table1=Column1, Table1=Column2, Table2=Column3". You can use the "*" character to include all tables and all columns.

 MyTable=*
SSL Server Cert

The certificate to be accepted from the server when connecting using TLS/SSL. You can specify any of the following:

  • full PEM certificate
  • path to a local file containing the certificate
  • the public key
  • the MD5 Thumbprint (hex values can also be either space or colon separated)
  • the SHA1 Thumbprint (hex values can also be either space or colon separated)

If using a TLS/SSL connection, this property can be used to specify the TLS/SSL certificate to be accepted from the server. Any other certificate that is not trusted by the machine will be rejected.

 C:\cert.cer
Support Enhanced SQL

Enhances SQL functionality beyond what can be supported through the API directly, by enabling in-memory client-side processing:

  • true the connector offloads as much of the SELECT statement processing as possible to IMAP and then processes the rest of the query in memory. In this way the driver can execute unsupported predicates, joins, and aggregation
  • false the connector limits SQL execution to what is supported by the IMAP API

Execution of predicates

The connector determines which of the clauses are supported by the data source and then pushes them to the source to get the smallest superset of rows that would satisfy the query. It then filters the rest of the rows locally. The filter operation is streamed, which enables the driver to filter effectively for even very large datasets.

Execution of Joins

The connector uses various techniques to join in memory. The driver trades off memory utilization against the requirement of reading the same table more than once.

Execution of Aggregates

The connector retrieves all rows necessary to process the aggregation in memory.

  
UID Mode If true, permanent message Ids are used instead of the default temporary Ids. false
Upper Case Identifiers Report all identifiers in uppercase, including table and column names. false
Proxy Authentication Scheme

The authentication type to use to authenticate to the ProxyServer proxy.

This value specifies the authentication type to use to authenticate to the HTTP proxy specified by ProxyServer and ProxyPort.

Note

The connector will use the system proxy settings by default, without further configuration needed; if you want to connect to another proxy, you will need to set ProxyAutoDetect to false, in addition to ProxyServer and ProxyPort. To authenticate, set ProxyAuthScheme and set ProxyUser and ProxyPassword, if needed.

The authentication type can be one of the following:

  • BASIC The driver performs HTTP BASIC authentication
  • DIGEST The driver performs HTTP DIGEST authentication.
  • NEGOTIATE The driver retrieves an NTLM or Kerberos token based on the applicable protocol for authentication
  • PROPRIETARY The driver does not generate an NTLM or Kerberos token. You must supply this token in the Authorization header of the HTTP request
 BASIC
Proxy Auto Detect Indicates whether to use the system proxy settings or not. Set ProxyAutoDetect to FALSE to use custom proxy settings. This takes precedence over other proxy settings. true
Proxy User

A user name to be used to authenticate to the ProxyServer proxy.

The ProxyUser and ProxyPassword options are used to connect and authenticate against the HTTP proxy specified in ProxyServer.

You can select one of the available authentication types in ProxyAuthScheme. If you are using HTTP authentication, set this to the username of a user recognized by the HTTP proxy. If you are using Windows or Kerberos authentication, set this property to a username in one of the following formats:

  • user@domain
  • domain\user
 john_doe@example.com
Proxy Password

A password to be used to authenticate to the ProxyServer proxy.

This property is used to authenticate to an HTTP proxy server that supports NTLM (Windows), Kerberos, or HTTP authentication. To specify the HTTP proxy, you can set ProxyServer and ProxyPort. To specify the authentication type, set ProxyAuthScheme.

If you are using HTTP authentication, additionally set ProxyUser and ProxyPassword to HTTP proxy.

If you are using NTLM authentication, set ProxyUser and ProxyPassword to your Windows password. You may also need these to complete Kerberos authentication

  
Proxy Server

The hostname or IP address of a proxy to route HTTP traffic through.

The hostname or IP address of a proxy to route HTTP traffic through. The driver can use the HTTP, Windows (NTLM), or Kerberos authentication types to authenticate to an HTTP proxy.

By default, the driver uses the system proxy. If you need to use another proxy, set ProxyAutoDetect to false.

 206.174.193.115
Proxy Port The TCP port the ProxyServer proxy is running on. 80
Proxy SSL Type

The SSL type to use when connecting to the ProxyServer proxy:

  • AUTO If the URL is an HTTPS URL, the connector will use the TUNNEL option. If the URL is an HTTP URL, the connector will use the NEVER option (default)
  • ALWAYS the connection is always SSL enabled
  • NEVER the connection is not SSL enabled
  • TUNNEL the connection is through a tunneling proxy: The proxy server opens a connection to the remote host and traffic flows back and forth through the proxy
 AUTO

Retrieving message bodies and attachments

By default, the message body and any attachments are only returned when you select one record from a table. If more than one record is returned, these fields are left blank.

If you want to retrieve the message body and attachments for more than one record, you must set the Include Message option in the Advanced Settings. Returning these fields is resource-intensive and doing so for a number of records affects performance. If you need to examine the message body or attachments, try using other fields to identify the messages you want to analyze in detail. Then query this subset of messages individually to examine the message body and attachments.

Data connector updates

When you upgrade Analytics, the Robots Agent, or AX Server, you should test any of your scripts that import data using one of the Analytics data connectors (ACCESSDATA command).

The possibility exists that changes made by third-party data sources or ODBC driver vendors required updates to one or more of the data connectors. Scripted data connections may need to be updated in order to continue working correctly.

  • Re-run the import The easiest way to update a connection is to manually perform an import using the Data Access window in the upgraded version of Analytics. Copy the ACCESSDATA command from the log and use it to update your script.

    Note

    Before connecting to a data source and re-running the import, clear the connector cache to flush the existing set of table names.

    In the Existing Connections tab in the Data Access window, beside the connector name, select > Clear cache.

  • Update field specifications You may also need to update field specifications in the script body to align with table schema changes in the data source or ODBC driver. Possible changes include field names, field data types, and field and record lengths.
  • Check the results of any filtering You should also check the results of any filtering that you apply as part of the data import. Confirm that the import filtering is including and excluding records correctly.

Email data connector changes

Specific changes made to the Email data connector are listed below.

Analytics version Change

14.2

Existing fields To and From now contain email address only.

New fields FullTo and FullFrom contain email address and email alias.
15.0

Default value for the Max Items field in the connector is -1. When this value is specified, the connector returns all items during import.

After upgrading to 15.0, if value for Max Items is specified as 100 or any other value, the connector returns items only from the specified number of records.

If you were using the ACCESSDATA command in a previous version of Analytics, when upgrading to 15.0, to return all items, open the script, update the value for maxitems to -1, and re-run the script.