In today's data-driven world, the ability to export data is crucial for various reasons, including backup, analysis, and integration with other systems. However, data export processes often come with limitations that users should be aware of. This article provides a detailed explanation of data export limitations, focusing on specific examples such as Salesforce and Power BI, and offers guidance on how to navigate these challenges.
Introduction
Data export is the process of extracting data from a source system and transferring it to another system or storage location. This is a common practice for organizations that need to back up their data, analyze it using different tools, or integrate it with other applications. However, data export processes are not always straightforward and can be subject to various limitations. Understanding these limitations is essential for ensuring data integrity, security, and compliance.
Data Export Limitations in Salesforce
Salesforce, a leading Customer Relationship Management (CRM) platform, offers several methods for exporting data, including the Data Export Service (Data Export Wizard), Data Loader, Dataloader.io, and Report Export. Each of these methods has its own set of limitations that users should be aware of.
Salesforce Data Export Service (Data Export Wizard)
The Data Export Service is a native Salesforce functionality that allows users to perform manual or scheduled exports of their data on a weekly (once every 7 days) or monthly (once every 29 days) basis. The data is exported as a set of comma-separated values (CSV) files.
Limitations:
- Frequency: Exports can only be scheduled weekly or monthly, which may not be frequent enough for organizations that require more regular backups. This means you will lose up to seven days' worth of data that was changed or deleted in between backups.
- Manual Setup: The export process has to be set up manually, which can be tedious and time-consuming. Every time you update your Salesforce records, you will have to repeat the export process.
- Data Selection: There is no option to select specific records or fields to be exported. All the records of a particular object will be exported, thus increasing the size of the export. Formula (derived) and roll-up summary fields are always excluded from exports.
- Queueing: Sometimes, your first request will still be in the queue when your next request is made, as specified in the Salesforce Help page.
- Sandbox Support: The data export service isn't supported in sandboxes. You can request an export in your sandbox, but the export doesn’t get processed and doesn’t complete.
- File Deletion: Exported files are deleted 48 hours after they're generated (not including weekends).
- File Size: If your organization has a large amount of data, multiple .zip archives are created. Each .zip archive contains one or more CSV files and can be up to 512 MB (approximately).
How to export data from Salesforce using Salesforce Data Export Service:
To export data using Data Export Service, follow the below steps:
Read also: Customized skincare explained
- Step 1: Go to Setup, enter “Data Export” in the Quick Find box, and select Data Export.
- Step 2: Click Export Now or Schedule Export. The Export Now option prepares your files for export immediately. The export files are also available on the same page for 48 hours, after which they are deleted. Large exports are usually broken into multiple files. Note: Including special content in the export increases data export processing time. If you want to have spaces instead of carriage returns or line breaks in your export files, select Replace carriage returns with spaces. This selection is useful if you plan to use your export files for importing or other integrations. When you have to select the types of data to include in your export under the option Exported Data, keep the following in mind:
- If your org uses divisions, data from all divisions is included in the export.
- If your org uses person accounts and you are exporting accounts, all account fields are included in the account data.
- If your org uses person accounts and you are exporting contacts, person account records are included in the contact data. However, the contact data only includes the fields shared by contacts and person accounts.
Salesforce Data Loader
Data Loader is a client application developed by Salesforce that needs to be installed separately. The application allows you to perform mass data operations like import, export, update, or delete in Salesforce. It can be operated either through the user interface or by generating the command line. The latter option is useful if you want to automate the data export process or integrate to another system through API.
Limitations:
- Installation Required: Needs to be installed separately on your system.
- Attachment Export: Does not support exporting attachments. Alternately, one can use the weekly export feature in the online application to export attachments.
- File Format: Data Loader can only produce output in CSV formats.
- Complexity: It takes multiple steps to complete the migration, taking a lot of time.
- Object Relationships: There is difficulty in creating object relationships automatically.
How to export data from Salesforce using Salesforce Data Loader:
To export Salesforce data using Data Loader, follow the below steps:
- Step 1: Go to Setup and search for Data Loader in the Quick Find box. Download the Data Loader installation file and install the application. See detailed steps on how to install Data Loader on Windows or macOS.
- Step 2: Open the Data Loader. Click Export. If you also want to export archived activity records and soft-deleted records, click Export All instead.
- Step 3: Enter your Salesforce username and password, and click Log in. When you are logged in, click Next. Note: If your organization restricts IP addresses, logins from untrusted IPs are blocked until they are activated.
- Step 4: Choose the object to be exported. If your object name is not listed, select Show all objects to see all the objects that you can access. The objects are listed by localized label name, with the developer name in parentheses.
- Step 5: Choose the CSV file to which the data needs to be exported. Click Next.
- Step 6: From the list of fields of the given object, choose the ones that need to be exported. You also have the option to add conditions to filter the records. If you do not select any conditions, all the data to which you have read access will be exported. An SOQL query will be auto generated based on your selections. Click Finish. Select Yes to confirm the export process.
- Step 7: To view the CSV file, click View Extraction. To open the CSV file in associated external program such as Microsoft Excel, click the button Open in external program.
Dataloader.io
Dataloader.io is a 100% cloud-based solution to import, export, update, and delete data in Salesforce. Originally developed by MuleSoft, dataloader.io has been brought into Salesforce family through Salesforce’s acquisition of MuleSoft. It can be accessed through your browser and has all the features available in Salesforce Data Loader, with some additional functionality such as the ability to schedule tasks, pull/push information from/into remote directories like FTP, Dropbox, and Box, and automatically find parent records based on text, not an ID.
Limitations:
- Cost: The free version has a limit of 10,000 records per month; Professional and Enterprise editions are very costly ($99/user/month and $299/user/month respectively).
- Scheduling: Only one task can be scheduled at a time in the free version.
- File Format: Supports only CSV file format.
How to export data from Salesforce using dataloader.io:
To export Salesforce data using dataloader.io, follow the below steps:
- Step 1: From Setup, enter Dataloader.io in the Quick Find box, then select Dataloader.io.
- Step 2: Click Launch dataloader.io.
- Step 3: Click Login with Salesforce. Uncheck the option I have a custom Salesforce domain and choose Production as the Environment. Check the option I agree to the Terms and Conditions and click Login. Click Allow.
- Step 4: In the dataloader.io application, click NEW TASK-> EXPORT.
- Step 5: Leave the Connection selection as it is and select the object that needs to be exported from the Objects list. For example, select Account. Click Next.
- Step 6: Select the fields that need to be exported. You can select the fields you want from the selected object and its related objects too. You can also filter the records to be exported by selecting the required conditions. You can also specify the order in which the records will be listed. An SOQL query will be auto generated based on your selections. Alternately, you can add a custom SOQL query to meet specific requirements. Click Next.
- Step 7: A summary of the export task will be displayed. You can choose to schedule the export hourly, daily, weekly, or monthly. By default, once an export task runs in the Data Loader, you can obtain its output by either clicking on the latest run displayed in the task manager, or by looking at the task's history; you then download the file through your browser. Alternately, you can select a location under the tab Add Destination Folder to send the exported data to. (Since Data Loader provides direct connectivity to FTP, SFTP, Dropbox, and Box.)
- Step 8: After selecting your specific requirements in Step 7, click Save & Run.
- Step 9: Once the task is completed, click on the link next to the Task Run number to download the CSV file containing your export.
Report Export
Report export is a simple way to export data out of Salesforce using the reports interface. To work with report data in a dedicated tool such as a spreadsheet, you can export report data as a Microsoft Excel (.xlsx or .xls) file or comma-separated values (.csv) file.
Read also: Is a Fruitarian Diet Healthy?
Limitations:
- Subscription Limit: Each user can only subscribe to five reports.
- Formatting Issues: When exporting a report in Lightning Experience as Formatted Report, report charts aren't included in the export.
- Data Volume: Large reports may be difficult to export and process.
How to export Salesforce data using Report Export:
To export reports in Salesforce, follow the below steps:
- Step 1: Navigate to the Reports tab.
- Step 2: Select the report you want to export.
- Step 3: Click the "Export" button.
- Step 4: Choose the export format (e.g., .xlsx, .csv).
- Step 5: Select the view:
- Formatted Report view exports the report as it appears in Salesforce, with the report header, groupings, and filter details.
- Details Only view exports each detail row without the formatting. This will send you a report on a daily, weekly, monthly, or condition-based basis with the report details.
- Step 6: Click "Export".
General Salesforce Data Export Considerations
Salesforce’s data export process has limitations that could lead to data vulnerabilities and loss:
- Salesforce offers manual and scheduled data exports, but these are infrequent and require active management to ensure completeness.
- The data export process can be time-consuming and may not cover all data, leaving gaps in protection and increasing the risk of data loss.
Given the large amount of data that is stored in Salesforce, it is the responsibility of IT administrators to ensure that data is stored securely in the cloud so that business data is always available. Even though Salesforce is a secure platform, data loss due to human error is inevitable. It is recommended that you keep a regular backup of your data and to do a manual point-in-time backup before you proceed with any major data project within your org.
Data Export Limitations in Power BI Desktop
Power BI Desktop is a powerful data visualization and analysis tool that allows users to connect to various data sources and create interactive reports. However, when using DirectQuery, there are limitations to the amount of data that can be exported.
DirectQuery Limitations
When using DirectQuery, the maximum amount of data that can be exported is 16 MB. Power BI only supports export in visuals that use basic aggregates. This limitation can be a significant challenge for users who need to export large datasets for further analysis or reporting.
Read also: Is Chicken Broth Diet Right for You?
Unicode Character Issues
If there is a Unicode character in the .csv file, the text in Excel may not display properly. Examples of Unicode characters are currency symbols and foreign words. The workaround for this is to import the CSV into Excel, instead of opening the CSV directly.
Continuous Export of Data from Kusto
Continuous export of data from Kusto to an external table involves running a query periodically and storing the results in the external table. This external table defines the destination, such as Azure Blob Storage, and the schema of the exported data.
Limitations and Considerations
- Distributed Mode: Continuous export runs in a distributed mode, where all nodes export concurrently. The number of artifacts depends on the number of nodes.
- Interval Between Runs: The continuous export runs according to the time period configured for it in the intervalBetweenRuns property. The recommended value for this interval is at least several minutes, depending on the latencies you're willing to accept. The intervalBetweenRuns serves as a recommendation only, and isn't guaranteed to be precise.
- File Partitioning: The number of files exported in each continuous export iteration depends on how the external table is partitioned.
- New Files Only: Each continuous export iteration always writes to new files, and never appends to existing ones. As a result, the number of exported files also depends on the frequency in which the continuous export runs.
- Storage Throttling: Continuous export works in a distributed manner, such that all nodes are exporting concurrently. On large databases, and if the exported data volume is large, this might lead to storage throttling. The recommendation is to configure multiple storage accounts for the external table.
- "Exactly Once" Guarantee: To guarantee "exactly once" export, continuous export uses database cursors. The continuous export query shouldn't include a timestamp filter - the database cursors mechanism ensures that records aren't processed more than once. IngestionTime policy must be enabled on all tables referenced in the query that should be processed "exactly once" in the export.
- Potential Duplicates: Continuous export doesn't guarantee that each record is written only once to the external table. If a failure occurs after export begins and some of the artifacts were already written to the external table, the external table might contain duplicates.
- Corrupted Files: If a write operation was aborted before completion, the external table might contain corrupted files. In such cases, artifacts aren't deleted from the external table, but they aren't reported in the show exported artifacts command.
- Fact and Dimension Tables: By default, all tables referenced in the export query are assumed to be fact tables. As such, they're scoped to the database cursor. The syntax explicitly declares which tables are scoped (fact) and which aren't scoped (dimension). When using joins between fact and dimension tables in continuous-export, keep in mind that records in the fact table are only processed once. If the export runs while records in the dimension tables are missing for some keys, records for the respective keys are either missed or include null values for the dimension columns in the exported files. Continuous export of only dimension tables isn't supported.
- Max Lateness: Max lateness (in minutes) of continuous exports in the database. This is the time between now and the min ExportedTo time of all continuous export jobs in database.
- Data Export Capacity: The number of export operations that can run concurrently is limited by the database's data export capacity.
- Starting Point: Continuous export starts exporting data only from the point of its creation. Records ingested before that time should be exported separately using the non-continuous export command. To avoid duplicates with data exported by continuous export, use StartCursor returned by the show continuous export command and export only records where cursorbeforeor_at the cursor value.
- Schema Synchronization: If the schema isn't provided, Kusto will try infer it automatically if there is already a delta table defined in the target storage container. The schema of the delta table must be in sync with the continuous export query.
- Ingestion Methods: Records in source table must be ingested to the table directly, using an update policy, or ingest from query commands. If records are moved into the table using .move extents or using .rename table, continuous export might not process these records.
- Cross-Database Calls: Continuous export supports cross-database calls only for dimension tables. All fact tables must reside in the local database.
- Row Level Security: Continuous export can't be enabled on a table with Row Level Security policy unless specific conditions are met.
Data Privacy and Security Regulations
Data export limitations are not only technical but also legal and regulatory. Companies must comply with data privacy and security regulations such as the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA). These regulations impose restrictions on the transfer of personal data to third countries or international organizations.
The Rule on Prohibited and Restricted Data Transactions
The Department of Justice (DOJ) has issued a final rule (the Rule) to address the national security risk arising from access by individuals and entities located in China and other designated countries to Americans’ sensitive personal data and United States Government-related data.
The Rule can be understood as introducing a presumptive ban on covered data transactions that takes place on or after April 8, 2025, even if the transaction is addressed by an agreement that predates the Rule’s effective date. sensitive data or government-related data to covered persons.
Prohibited Transactions
Prohibited transactions include commercial transactions that result in the transfer of bulk sensitive personal data to social media applications owned by a country of concern. Such transactions are prohibited without exception. For example, if a company operates a mobile application and knowingly incorporates into the mobile application certain tracking pixels or software development kits that in turn transfer bulk sensitive personal data to social media applications owned by a country of concern, that transaction is prohibited.
Restricted Transactions
Restricted transactions include commercial transactions that constitute “covered data transactions,” which may include employment agreements, vendor agreements and investment agreements. Restricted transactions may proceed only if certain security measures designed to prevent or minimize access by persons located in countries of concern are applied, including organization, system and data-level protections. For example, if a company collects personal data from U.S. users through a mobile application and then shares that data with a vendor located in a country of concern must comply with specific security protocols to ensure the data remains protected. Specifically, a restricted transaction is only permitted if the entity applies the security requirements outlined in , which took effect in January 2025, in a manner that prevents access by individuals in the countries of concern.
Exemptions
Exempt from the Rule are a variety of transactions such as personal communications; telecommunications services; travel; financial services; drug, biological product and medical device authorizations; transactions necessary for compliance with federal law; and some corporate group transactions.
Penalties and Enforcement
The Rule contemplates sweeping enforcement authority for the DOJ through audits, civil investigative demands and criminal inquiries. Non-compliance with the Rule could lead to significant penalties, including civil penalties up to $377,700 per violation, or double the value of the covered transaction. Willful violations of the Rule can bring criminal fines of up to $1 million and up to two years in prison.
Before imposing penalties, the DOJ will notify alleged violators through a pre-penalty notice process, which will provide the alleged violators an opportunity to respond.
Compliance
Companies should work with legal counsel to assess whether and to what extent they may be involved in covered data transactions with individuals or entities located in China and other countries of concern. In many cases, this will require a close evaluation of data supply chains among data processors and subprocessors to ensure all instances of potential access by individuals located in countries of concern are addressed.
Overcoming Data Export Limitations
Despite the various limitations associated with data export, there are several strategies that organizations can employ to overcome these challenges:
- Choose the Right Tool: Select the data export tool that best meets your specific needs and requirements. Consider factors such as data volume, frequency of export, file format, and cost.
- Optimize Data Selection: Where possible, select only the data that is necessary for your intended use. This can reduce the size of the export file and improve performance.
- Implement Data Compression: Use data compression techniques to reduce the size of the export file. This can make it easier to transfer and store the data.
- Automate the Export Process: Automate the data export process to reduce the risk of human error and ensure that exports are performed on a regular basis.
- Use Third-Party Backup Solutions: Consider using third-party cloud backup solutions that offer more frequent and comprehensive backups than the native Salesforce Data Export Service.
- Comply with Data Privacy Regulations: Ensure that your data export processes comply with all applicable data privacy regulations, such as GDPR and CCPA.
tags: #data #export #limitations