Hands-On Salesforce Data Cloud

Chapter 6. Data Ingestion and Storage

In the previous chapter, we were introduced to 23 different Data Cloud menu options that we categorized into five core capabilities. This chapter is a deep dive into the first one, the data ingestion and storage capabilities of Salesforce Data Cloud. We’ll be exploring each of the three menu options associated with the following data ingestion and storage capabilities:

• Data Explorer
• Data Streams
• Data Lake Objects

We’ll first learn about the Data Explorer tool since it will be referenced throughout the chapter, and we’ll then learn about creating data streams. As part of building out data streams, you’ll observe some Data Cloud object types. In Data Cloud, object types include DSOs, DLOs, DMOs, unstructured data lake objects (UDLOs), unstructured data model objects (UDMOs), and external data lake objects. Chapter 2 provides a detailed explanation of these Data Cloud object types. Most of the discussion in this chapter focuses on data ingestion and storage of structured data. We will cover how to handle unstructured data in Chapter 13.

Creation of Data Lake Objects

When using the Salesforce CRM Connector, the necessary DLOs can be created automatically. When using the other connectors for structured data, like the Amazon S3 Storage Connector, you’ll need to manually create a new DLO. UDLOs need to be manually created to access unstructured data, and they are always automatically mapped to a UDMO.

Once data is ingested and stored in DLOs, ...