Improving Your Splunk Skills

Improving Your Splunk Skills

by James D. Miller, Paul R. Johnson, Josh Diakun, Derek Mock
Released August 2019
Publisher(s): Packt Publishing
ISBN: 9781838981747

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Transform machine-generated data into valuable business insights using the powers of Splunk

Key Features

  • Explore the all-new machine learning toolkit in Splunk 7.x
  • Tackle any problems related to searching and analyzing your data with Splunk
  • Get the latest information and business insights on Splunk 7.x

Book Description

Splunk makes it easy for you to take control of your data and drive your business with the cutting edge of operational intelligence and business analytics. Through this Learning Path, you'll implement new services and utilize them to quickly and efficiently process machine-generated big data.

You'll begin with an introduction to the new features, improvements, and offerings of Splunk 7. You'll learn to efficiently use wildcards and modify your search to make it faster. You'll learn how to enhance your applications by using XML dashboards and configuring and extending Splunk. You'll also find step-by-step demonstrations that'll walk you through building an operational intelligence application. As you progress, you'll explore data models and pivots to extend your intelligence capabilities.

By the end of this Learning Path, you'll have the skills and confidence to implement various Splunk services in your projects.

This Learning Path includes content from the following Packt products:

  • Implementing Splunk 7 - Third Edition by James Miller
  • Splunk Operational Intelligence Cookbook - Third Edition by Paul R Johnson, Josh Diakun, et al

What you will learn

  • Master the new offerings in Splunk: Splunk Cloud and the Machine Learning Toolkit
  • Create efficient and effective searches
  • Master the use of Splunk tables, charts, and graph enhancements
  • Use Splunk data models and pivots with faster data model acceleration
  • Master all aspects of Splunk XML dashboards with hands-on applications
  • Apply ML algorithms for forecasting and anomaly detection
  • Integrate advanced JavaScript charts and leverage Splunk's API

Who this book is for

This Learning Path is for data analysts, business analysts, and IT administrators who want to leverage the Splunk enterprise platform as a valuable operational intelligence tool. Existing Splunk users who want to upgrade and get up and running with Splunk 7.x will also find this book useful. Some knowledge of Splunk services will help you get the most out of this Learning Path.

Table of contents

  1. Title Page
  2. Copyright and Credits
    1. Improving Your Splunk Skills
  3. About Packt
    1. Why subscribe?
    2. Packt.com
  4. Contributors
    1. About the authors
    2. Packt is searching for authors like you
  5. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
      1. Download the example code files
      2. Conventions used
    4. Get in touch
      1. Reviews
  6. The Splunk Interface
    1. Logging in to Splunk
    2. The home app
    3. The top bar
    4. The Search & Reporting app
      1. Data generator
      2. The Summary view
      3. Search
      4. Actions
      5. Timeline
      6. The field picker
        1. Fields
      7. Search results
        1. Options
        2. Events viewer
    5. Using the time picker
    6. Using the field picker
    7. The settings section
    8. Splunk Cloud
    9. Try before you buy
    10. A quick cloud tour
    11. The top bar in Splunk Cloud
    12. Splunk reference app – PAS
    13. Universal forwarder
    14. eventgen
    15. Next steps
  7. Understanding Search
    1. Using search terms effectively
    2. Boolean and grouping operators
    3. Clicking to modify your search
      1. Event segmentation
      2. Field widgets
      3. Time
    4. Using fields to search
      1. Using the field picker
    5. Using wildcards efficiently
      1. Supplementing wildcards in fields
    6. All about time
      1. How Splunk parses time
      2. How Splunk stores time
      3. How Splunk displays time
      4. How time zones are determined and why it matters
      5. Different ways to search against time
        1. Presets
        2. Relative
        3. Real-time
          1. Windowed real-time versus all-time real-time searches
        4. Date range
        5. Date and time range
        6. Advanced
      6. Specifying time in-line in your search
        1. _indextime versus _time
    7. Making searches faster
    8. Sharing results with others
      1. The URL
      2. Save As Report
      3. Save As Dashboard Panel
      4. Save As Alert
      5. Save As Event Type
    9. Searching job settings
    10. Saving searches for reuse
    11. Creating alerts from searches
      1. Enable Actions
      2. Action Options
      3. Sharing
    12. Event annotations
      1. An illustration
  8. Tables, Charts, and Fields
    1. About the pipe symbol
    2. Using top to show common field values
      1. Controlling the output of top
    3. Using stats to aggregate values
    4. Using chart to turn data
    5. Using timechart to show values over time
      1. The timechart options
    6. Working with fields
      1. A regular expression primer
      2. Commands that create fields
        1. eval
        2. rex
      3. Extracting loglevel
        1. Using the extract fields interface
        2. Using rex to prototype a field
        3. Using the admin interface to build a field
        4. Indexed fields versus extracted fields
          1. Indexed field case 1 - rare instances of a common term
          2. Indexed field case 2 - splitting words
          3. Indexed field case 3 - application from source
          4. Indexed field case 4 - slow requests
          5. Indexed field case 5 - unneeded work
    7. Chart enhancements in version 7.0
      1. charting.lineWidth
      2. charting.data.fieldHideList
      3. charting.legend.mode
      4. charting.fieldDashStyles
      5. charting.axis Y.abbreviation
  9. Data Models and Pivots
    1. What is a data model?
    2. What does a data model search?
      1. Data model objects
        1. Object constraining
        2. Attributes
    3. Acceleration in version 7.0
    4. Creating a data model
      1. Filling in the new data model dialog
      2. Editing fields (attributes)
    5. Lookup attributes
      1. Children
    6. What is a pivot?
      1. The Pivot Editor
      2. Working with pivot elements
        1. Filtering pivots
      3. Split (row or column)
        1. Column values
      4. Pivot table formatting
    7. A quick example
    8. Sparklines
  10. Simple XML Dashboards
    1. The purpose of dashboards
    2. Using wizards to build dashboards
      1. Adding another panel
        1. A cool trick
    3. Converting the panel to a report
      1. More options
    4. Back to the dashboard
      1. Add input
      2. Editing source
      3. Edit UI
    5. Editing XML directly
    6. UI examples app
    7. Building forms
      1. Creating a form from a dashboard
      2. Driving multiple panels from one form
      3. Post-processing search results
      4. Post-processing limitations
    8. Features replaced
    9. Autorun dashboard
    10. Scheduling the generation of dashboards
  11. Extending Search
    1. Using tags to simplify search
    2. Using event types to categorize results
    3. Using lookups to enrich data
      1. Defining a lookup table file
      2. Defining a lookup definition
      3. Defining an automatic lookup
      4. Troubleshooting lookups
    4. Using macros to reuse logic
      1. Creating a simple macro
      2. Creating a macro with arguments
    5. Creating workflow actions
      1. Running a new search using values from an event
      2. Linking to an external site
      3. Building a workflow action to show field context
        1. Building the context workflow action
        2. Building the context macro
    6. Using external commands
      1. Extracting values from XML
        1. xmlkv
        2. XPath
      2. Using Google to generate results
  12. Working with Apps
    1. Defining an app
    2. Included apps
    3. Installing apps
      1. Installing apps from Splunkbase
        1. Using Geo Location Lookup Script
        2. Using Google Maps
      2. Installing apps from a file
    4. Building your first app
    5. Editing navigation
    6. Customizing the appearance of your app
      1. Customizing the launcher icon
      2. Using custom CSS
      3. Using custom HTML
        1. Custom HTML in a simple dashboard
        2. Using server-side include in a complex dashboard
    7. Object permissions
      1. How permissions affect navigation
      2. How permissions affect other objects
      3. Correcting permission problems
    8. App directory structure
      1. Adding your app to Splunkbase
        1. Preparing your app
        2. Confirming sharing settings
        3. Cleaning up our directories
      2. Packaging your app
      3. Uploading your app
    9. Self-service app management
  13. Building Advanced Dashboards
    1. Reasons for working with advanced XML
    2. Reasons for not working with advanced XML
    3. Development process
    4. Advanced XML structure
    5. Converting simple XML to advanced XML
    6. Module logic flow
    7. Understanding layoutPanel
      1. Panel placement
    8. Reusing a query
    9. Using intentions
      1. stringreplace
      2. addterm
    10. Creating a custom drilldown
      1. Building a drilldown to a custom query
      2. Building a drilldown to another panel
      3. Building a drilldown to multiple panels using HiddenPostProcess
    11. Third-party add-ons
      1. Google Maps
      2. Sideview Utils
      3. The Sideview search module
        1. Linking views with Sideview
        2. Sideview URLLoader
        3. Sideview forms
  14. Summary Indexes and CSV Files
    1. Understanding summary indexes
      1. Creating a summary index
    2. When to use a summary index
    3. When to not use a summary index
    4. Populating summary indexes with saved searches
    5. Using summary index events in a query
    6. Using sistats, sitop, and sitimechart
    7. How latency affects summary queries
    8. How and when to backfill summary data
      1. Using fill_summary_index.py to backfill
      2. Using collect to produce custom summary indexes
    9. Reducing summary index size
      1. Using eval and rex to define grouping fields
      2. Using a lookup with wildcards
      3. Using event types to group results
    10. Calculating top for a large time frame
      1. Summary index searches
    11. Using CSV files to store transient data
      1. Pre-populating a dropdown
      2. Creating a running calculation for a day
  15. Configuring Splunk
    1. Locating Splunk configuration files
    2. The structure of a Splunk configuration file
    3. The configuration merging logic
      1. The merging order
        1. The merging order outside of search
        2. The merging order when searching
      2. The configuration merging logic
        1. Configuration merging – example 1
        2. Configuration merging – example 2
        3. Configuration merging – example 3
        4. Configuration merging – example 4, search
      3. Using btool
    4. An overview of Splunk.conf files
      1. props.conf
        1. Common attributes
          1. Search-time attributes
          2. Index-time attributes
          3. Parse-time attributes
          4. Input-time attributes
        2. Stanza types
        3. Priorities inside a type
        4. Attributes with class
      2. inputs.conf
        1. Common input attributes
        2. Files as inputs
          1. Using patterns to select rolled logs
          2. Using blacklist and whitelist
          3. Selecting files recursively
          4. Following symbolic links
          5. Setting the value of the host from the source
          6. Ignoring old data at installation
          7. When to use crcSalt
          8. Destructively indexing files
        3. Network inputs
        4. Native Windows inputs
        5. Scripts as inputs
      3. transforms.conf
        1. Creating indexed fields
          1. Creating a loglevel field
          2. Creating a session field from the source
          3. Creating a tag field
          4. Creating host categorization fields
        2. Modifying metadata fields
          1. Overriding the host
          2. Overriding the source
          3. Overriding sourcetype
          4. Routing events to a different index
        3. Lookup definitions
          1. Wildcard lookups
          2. CIDR wildcard lookups
          3. Using time in lookups
        4. Using REPORT
          1. Creating multivalue fields
          2. Creating dynamic fields
        5. Chaining transforms
        6. Dropping events
      4. fields.conf
      5. outputs.conf
      6. indexes.conf
      7. authorize.conf
      8. savedsearches.conf
      9. times.conf
      10. commands.conf
      11. web.conf
    5. User interface resources
      1. Views and navigation
      2. Appserver resources
      3. Metadata
  16. Play Time – Getting Data In
    1. Introduction
    2. Indexing files and directories
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Adding a file or directory data input using the CLI
        2. Adding a file or directory input using inputs.conf
        3. One-time indexing of data files using the Splunk CLI
        4. Indexing the Windows event logs
      5. See also
    3. Getting data through network ports
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Adding a network input using the CLI
        2. Adding a network input using inputs.conf
      5. See also
    4. Using scripted inputs
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. See also
    5. Using modular inputs
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
      5. See also
    6. Using the Universal Forwarder to gather data
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Adding the receiving indexer via outputs.conf
    7. Receiving data using the HTTP Event Collector
      1. Getting ready
      2. How to do it...
      3. How it works...
    8. Getting data from databases using DB Connect
      1. Getting ready
      2. How to do it...
      3. How it works...
    9. Loading the sample data for this book
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. See also
    10. Data onboarding – defining field extractions
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. See also
    11. Data onboarding - defining event types and tags
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Adding event types and tags using eventtypes.conf and tags.conf
      5. See also
    12. Installing the Machine Learning Toolkit
      1. Getting ready
      2. How to do it...
      3. How it works...
  17. Building an Operational Intelligence Application
    1. Introduction
    2. Creating an Operational Intelligence application
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Creating an application from another application
        2. Downloading and installing a Splunk app
      5. See also
    3. Adding dashboards and reports
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Changing permissions of saved reports
      5. See also
    4. Organizing the dashboards more efficiently
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Modifying the Simple XML directly
      5. See also
    5. Dynamically drilling down on activity reports
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Disabling the drilldown feature in tables and charts
      5. See also
    6. Creating a form for searching web activity
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Adding a Submit button to your form
      5. See also
    7. Linking web page activity reports to the form
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Adding an overlay to the Sessions Over Time chart
      5. See also
    8. Displaying a geographical map of visitors
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Adding a map panel using Simple XML
        2. Mapping different distributions by area
      5. See also
    9. Highlighting average product price
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. See also
    10. Scheduling the PDF delivery of a dashboard
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. See also
  18. Diving Deeper – Advanced Searching, Machine Learning and Predictive Analytics
    1. Introduction
      1. Identifying and grouping transactions
      2. Converging data sources
      3. Identifying relationships between fields
      4. Predicting future values
      5. Discovering anomalous values
      6. Leveraging machine learning
    2. Calculating the average session time on a website
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Starts with a website visit, ends with a checkout
        2. Defining maximum pause, span, and events in a transaction
      5. See also
    3. Calculating the average execution time for multi-tier web requests
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Calculating the average execution time without using a join
      5. See also
    4. Displaying the maximum concurrent checkouts
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. See also
    5. Analyzing the relationship of web requests
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Analyzing relationships of DB actions to memory utilization
      5. See also
    6. Predicting website traffic volumes
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Create and apply a machine learning model of traffic over time
        2. Predicting the total number of items purchased
        3. Predicting the average response time of function calls
      5. See also
    7. Finding abnormally-sized web requests
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. The anomalies command
        2. The anomalousvalue command
        3. The anomalydetection command
        4. The cluster command
      5. See also
    8. Identifying potential session spoofing
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Creating logic for urgency
      5. See also
    9. Detecting outliers in server response times
      1. Getting ready
      2. How to do it...
      3. How it works...
    10. Forecasting weekly sales
      1. Getting ready
      2. How to do it...
      3. How it works...
  19. Speeding Up Intelligence – Data Summarization
    1. Introduction
      1. Data summarization
      2. Data summarization methods
        1. About summary indexing
          1. How summary indexing helps
        2. About report acceleration
          1. The simplicity of report acceleration
    2. Calculating an hourly count of sessions versus completed transactions
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Generating the summary more frequently
        2. Avoiding summary index overlaps and gaps
      5. See also
    3. Backfilling the number of purchases by city
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Backfilling a summary index from within a search directly
      5. See also
    4. Displaying the maximum number of concurrent sessions over time
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Viewing the status of an accelerated report and how 
      5. See also
  20. Above and Beyond – Customization, Web Framework, HTTP Event Collector, REST API, and SDKs
    1. Introduction
      1. Web framework
      2. REST API
      3. Software development kits (SDKs)
      4. HTTP Event Collector (HEC)
    2. Customizing the application navigation
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
    3. Adding a Sankey diagram of web hits
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Changing the Sankey diagram options
      5. See also
    4. Developing a tag cloud of purchases by country
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's More...
      5. See also
    5. Adding Cell Icons to Highlight Average Product Price
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. See also
    6. Remotely querying Splunk's REST API for unique page views
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Authenticating with a session token
      5. See also
    7. Creating a Python application to return unique IP addresses
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Paginating the results of your search
      5. See also
    8. Creating a custom search command to format product names
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. See also
    9. Collecting data from remote scanning devices
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. See also
  21. Other Books You May Enjoy
    1. Leave a review - let other readers know what you think

Product information

  • Title: Improving Your Splunk Skills
  • Author(s): James D. Miller, Paul R. Johnson, Josh Diakun, Derek Mock
  • Release date: August 2019
  • Publisher(s): Packt Publishing
  • ISBN: 9781838981747