Day 3 - MSP Bootcamp Training 201
Day 3 - MSP Bootcamp Training 201
2 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Module 3 Objectives
By the end of the module, you will be able to explain the different
authentication methods supported by Zscaler.
● Explain how authentication and user provisioning works.
● Explain Location and Sub-location.
● Explain how cookie based authentication works.
● Describe user search, group search, and advanced search filters.
● Explain ZAB(Zscaler Auth Bridge).
● Explain SAML auth and integrate OKTA with your ZIA instance.
● Explain the concepts of Multiple IDP.
● Explain Admin authentication with SAML.
3 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Authentication And User Provisioning
User authentication to the platform
●Its absolute must that this step is completed before we proceed any
further.
● https://www.youtube.com/watch?v=4fVi0_iFEJ4
4 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Authentication And User Provisioning
User authentication to the platform
5 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Key points to understand before discussing Auth.
Location
● What is Location:
● Locations identify the various networks from which your organization sends its internet traffic.
● Location helps Zscaler identify which company does the user belong to.
● Following types can be provisioned on Zscaler as location:
● Customer Public IP Addresses.
● VPN Credentials.
● Dedicated proxy ports.
● Virtual ZEN’s or Virtual ZEN Clusters.
6 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Key points to understand before discussing Auth.
Sub-Location
● What is Sub-Location:
● Sub-locations enable an organization to create new locations that reference IP addresses that are
encapsulated within a GRE or IPSec tunnel, or that are passed to the Zscaler service through X-
Forwarded-For (XFF) headers.
• For example, an organization can define a sub-location for its corporate network, and another sub-
location for its guest network, even if their traffic goes through the same GRE or IPSec tunnel. The
organization can then use these sub-locations to do the following:
• Implement different policies based on IP addresses.
• Enforce authentication on the internal corporate network, while disabling it for the guest network
• Provide reporting information for different internal networks/offices when they share the same
egress IP address
● What is provisioning:
● To authenticate any user to use Zscaler services, we need to provision that user onto Zscaler DB.
● This is called provisioning.
● We cannot authenticate any user who is not provisioned on Zscaler DB.
● We will cover different ways to provision a user(Hosted, Active Directory, OpenLDAP).
● What is authentication:
● Once the user is provisioned on Zscaler, we require the user to authenticate before any traffic is passed.
● There are multiple authentication methods supported by Zscaler(Hosted, SAML).
8 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Authentication And User Provisioning
User provisioning to the platform
● Active Directory
● AD/LDAP sync 🡪 Requires an inbound allow rule from SMCA to customer AD servers.
● ZAB 🡪 This component should be hosted on customer premises, thus avoiding the need to
have an inbound connection to customer AD servers.
● Open LDAP
● Open-source implementation of the Lightweight Directory Access Protocol.
9 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Authentication And User Provisioning
User Authentication to the platform
10 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Cookie Auth redirections for hosted DB authentication
Client/Browser SME/ZEN CA Web Server
www.domain.com
Password Form
Submitting password to CA
Cookie Auth redirections for hosted DB authentication
Client/Browser SME/ZEN CA Web Server
Redirection to SME if PW is valid with a ticket
Bind success
Submitting password to CA
Cookie Auth redirections for LDAP based user auth
Client/Browser SME/ZEN CA AD Web Server
User search and LDAP bind
using user cred
Rdr to SME if LDAP bind with usr cred is success
Carrying the token set Bind success
by CA
Inserts auth cookie and
redirect to webpage url
Website url with
token
SME inserts domain
cookie and redirects
Actual webpage url with Invalidates the zscaler cookie and send the
domain cookie HTTP request towards the website
● All your email domains should be added to your Zscaler account. The service synchronizes data only from
the configured domains.
● The Distinguished Name of a user with permission to bind to (or query) the directory server. The account
doesn't require privileged access (Bind DN).
● Search filter 🡪 Zscaler queries your AD based on this filter for all the users in AD. Used at the time of
provisioning.
● User search filter 🡪 While provisioning users, this is the search query sent to AD.
● Group search filter 🡪 While provisioning groups, this is the search query sent to AD.
● Advanced Search Filter 🡪 While doing BIND for user auth, this is the query sent for user search.
● Attributes synced:
● login name
● display Name
● Group
● Department names.
Search Filters(User and Group Search Filters)
● When Zscaler starts provisioning users and groups onto Zscaler, it follows the following sequence:
● BIND to AD using credentials provided.
● Query AD server using group search filter. This will help provision all groups onto Zscaler.
● Now query AD for all users using the user search filter. While querying, CA asks for following attributes as configured in
UI.
● After provisioning the users, at the time of user auth, we leverage advanced search filter if configured.
● Before we go in detail, one important thing to understand regarding BIND is that, for the BIND to happen, always the object
name must be in DN format. Ex: 🡪 CN=prajith av.,CN=Users,DC=avethan,DC=co,DC=in. This is DN of a demo user
[email protected].
● Now that we are clear on above, lets understand what happens when user enters password during user auth during AD auth.
● Once the user enters the password, Zscaler users the advanced search filter to search the user in AD database.
Search Filters(Advanced Search Filter)
● You can clearly see that here the search is more a wildcard search.
● So if [email protected] is authenticating and you do not configure proper advanced search filter, it can return:
● [email protected]
● [email protected]
● [email protected], etc
● Which causes issues with auth as unique user is not identified to BIND.
Enabling Secondary Directory
● This option can be enabled, to synchronize with two directory servers that have different sets of user data.
● We should never enable this option to synchronize the service with two servers that have duplicate information.
● If you enable this option, The Directory #1 and Directory #2 tabs will appear below.
AD sync limitations
● The Zscaler service can synchronize user information from one forest in an organization. If your organization
has multiple forests, you can set up a Global Catalog and allow Zscaler to connect to that AD server.
Otherwise, Zscaler recommends integrating with SAML.
● The Zscaler service can synchronize data from multiple domains, if they are all registered with Zscaler
Support.
● The Zscaler service doesn't support nested groups. You must manually identify specific group names within
each nested group and add them to the Group filter.
Troubleshooting authentication failure issue with AD/LDAP
bind
● The error code is important to note during the issue.
● The credential validation is not performed by CA it is validated against the AD so it will be a good
idea to test the credentials against the AD directly.
● If the user credentials are working fine with AD but not via Zscaler then the user filter and search
filter could be the issue, or the login attribute could be the issue.
● If possible, then push the communication port to plain text port like 389 or 3268 and collect a pcap
on the AD server when the user attempts to authenticate.
Lab Exercise #1
Issue: Customer using LDAP for authentication, but a user fails to login
• Problem Scenario:
a.Customer organization is using LDAP to authenticate its users to the Zscaler service. One user
has issues authenticating which is giving them an error at the login screen. Header traces are
below.
b.Header trace –
• Task:
a.Download the header trace and open it up using Fiddler.
b.Analyze the header trace and determine where the problem might be.
c.From your analysis, how would you communicate the results to the customer?
d.What would you advise the customer as next steps?
24 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Authentication - ZAB
25 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
ZAB(Zscaler Auth Bridge)
● The Zscaler Authentication Bridge (ZAB) is a virtual appliance that you can use to provision as well as
authenticate users.
● You can use the ZAB to automatically import user information from an Active Directory (AD) or a Lightweight
Directory Access Protocol (LDAP) server to the Zscaler database, without requiring inbound connections to
your directory server.
● The ZAB can be used solely as a provisioning tool in conjunction with another authentication mechanism,
such as SAML or Kerberos. Alternatively, it can be used for authentication using LDAP with SSL client
certificates.
● ZAB opens a long-living secure outbound tunnel to the Zscaler Central Authority (CA).the ZAB opens a long-
living secure outbound tunnel to the Zscaler Central Authority (CA). So, unlike AD, there is no inbound
connection needed to AD server here.
● It downloads the authentication profile configuration of your organization from the CA and connects to the
directory server. It synchronizes user information from the directory server to the Zscaler cloud on demand
or as scheduled.
ZAB: Provisioning
● You must allow inbound connection from ZAB IP, which is private IP.
29 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
SAML Overview
● Identity Provider:
● Performs authentication and passes the user's identity and authorization level to the service
provider.
● Service Provider:
● Trusts the identity provider and authorizes the given user to access the requested resource.
• Major Components
● Service Provider
● Identity Provider
● Client
Benefits of SAML
● Improved User Experience:
● SAML enables single sign-on by allowing users to authenticate at an identity provider
and then access service providers without additional authentication.
● Platform Neutrality:
● SAML abstracts the security framework away from platform architectures and particular
vendor implementations.
● Making security more independent of application logic is an important tenet of Service
Oriented Architecture.
Submit username to CA
CA provides SAML request
User submits SAML request to Idp.
Cookie Auth redirections for SAML auth
Client/Browser SME/ZEN CA IDP Web Server
● In IdP-initiated SAML, a user can log in directly from an SSO provider's portal by clicking the
Zscaler application icon.
● When the user clicks the Zscaler application icon, the IdP generates a SAML response that is
posted to Zscaler at
https://login.<Zscaler Cloud Name>:443/sso_upd/<organization_id>.
● The service obtains the login name and optionally the group, department, and username from
the SAML response.
SAML and Auto-Provisioning.
● You can enable SAML auto-provisioning to allow the service to automatically retrieve information
related to users, groups, and departments from the SAML response and automatically add the
information to the database.
● It can also automatically update a user's group membership based on the information retrieved from
the SAML response.
● If the user doesn't exist in the database, the user is added in the database along with the
group and the department values. This new user is activated, and all relevant policies are
enforced.
● If the user exists in the database, the user display name, group, and department values in the
SAML Response are updated in the database.
● If the user display name, group, and department values don't exist in the SAML response,
then these values are removed from the database too.
Zscaler and SAML
● Zscaler support SAML 2.0
• When creating a new IdP, you must specify at least one user authentication domains.
• This restriction doesn't apply to the default IdP. The default IdP is automatically assigned to all
domains that aren't associated with an IdP.
• Once a domain is mapped to a IdP, then all user provisioning and auth for that domain, must
happen using that IdP.
Multiple IDP
• When a domain is mapped to a IdP, then only that IdP
can auth & provision the users for said domain.
• Problem Scenario:
a.Customer organization is using SAML to authenticate its users to the Zscaler service. Their
authentication frequency is set to once or forever from the Admin portal. One user has issues
getting challenged with authentication everyday.
b.Header trace -
• Task:
a.Download the header trace and open it up using Fiddler.
b.Analyze the header trace and determine where the problem might be.
c.From your analysis, how would you communicate the results to the customer?
d.What would you advise the customer as next steps?
44 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Lab Exercise #2
Issue: User being redirected to incorrect IdP.
• Problem Scenario:
a.Customer organization is using SAML to authenticate its users to the Zscaler service. Customer
has configured multiple IdP’s. Their IdP config is shown below.
• Task:
a.Any user from location is being redirect to the default IdP rather than the configured IdP(ADFS).
b.All road warrior users get redirected to correct IdP upon entering the
username([email protected]).
c.Please help customer troubleshoot this issue.
45 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Authentication Frequency
46 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Authentication Frequency
• All of these settings apply only for cookie based auth mechanisms.
47 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Authentication Frequency
Recap of cookies inserted by Zscaler
• Auth cookie:
• Valid for: gateway.zscaler.net
• Sent only on HTTPS
• Indicates if user is authenticated to Zscaler or not
• Cleared when user is logged out
• Domain cookie
• Valid for: specific site
• Purpose:
• Prevent redirection for each access
• Valid till: Session only
48 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Authentication Frequency
User resuming here will User resuming here will
Daily Authentication not be prompted for auth be prompted for auth
• Encryption key is generated every 12 hours. This key is used to generate the cookie
• Generation time is not aligned to 12 O'clock or any specific time
• Maintenance window, downtime etc. can delay the generation
• Current generation time is 02:12 GMT and 14:12 GMT
• When we see an HTTP Request, we verify the cookie …
• Try to decrypt the cookie using the latest generated key
• Try to decrypt the cookie using the previous generated key
• If it succeeds, replace the cookie with old key with new cookie with latest key
• If both fail, then challenge user for authentication
• Theoretical Maximum user idle time without authentication: 23:59:59 hours
• Theoretical Minimum user idle time without authentication: 12:00:01 hours
49 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Authentication Frequency
Daily Authentication
50 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Auth Frequency
Custom Auth Frequency
51 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Custom Auth Frequency
Only Once
52 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Custom Auth Frequency
Once Per Session
53 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Admin Authentication
54 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Admin Authentication
Administrator Provisioning
• For adding administrator, you would go to: Administration 🡪 Administration Management 🡪 Add
Administrator
• You can customize Admin user password expiry under: Administration > Administrator Management.
> Password Expiration.
• The default is 180 days. The days can range from 15 to 365.
55 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Admin Authentication
[email protected]
Administrator Authentication
• If customer wants their IDP to authenticate admin users, they can configure the same under:
Administration > Administrator Management. > Administrator Management > SAML Authentication for
Administrators
• NameID must match an already provisioned user to be able to login via SAML auth.
56 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Break
57 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Virtual Machines (VZEN, NSS,ZAB)
58 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
ZIA Service Edge Introduction
• All the same functionality offered by Zscaler Enforcement Nodes deployed closest to the user in customers’ DC/premise
• Monitored, managed, maintained by Zscaler as an extension of the Zscaler Cloud Enforcement plane in customer’s premise
• Consistent Policy follows the user – no separate configuration required
Control plane
Policy Definition and Administration
Customer Datacenter
IDP
IDP
USA EU Private
Nanolog
streaming
59 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION
Logging plane SIEM
SECURING YOUR DIGITAL TRANSFORMATION
Benefits of ZIA Service Edge
60 ©2019 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION Securing your cloud transformation
Use Cases
Regulation : ZIA Service Edge is recommended when regulatory requirements restrict the
use of Zscaler public datacenters.
Maintaining Source IP: For applications and services requiring a dedicated egress IP
address
Best Practice: ZIA Service Edge should be deployed for high bandwidth networks (2Gbps and above)
61 ©2019 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION Securing your cloud transformation
ZIA Service Edge – Deployment Recommendations
• Installation and Initial Configuration by customer • Physical Installation and cabling by Customer
• Health Monitoring by Customer • Setup and Configuration by Zscaler Ops
• No Public IP addresses required • Health Monitoring, Upgrades, Updates, patching and
maintenance by Zscaler Ops
• Requires to provide Public IP address
62 ©2019 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION Securing your cloud transformation
Virtual Service Edge
Virtualized Platform
63 ©2019 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION Securing your cloud transformation
Crypto Hardware
64 ©2019 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION Securing your cloud transformation
Setting up a Virtual Service Edge
Admin UI: Create Virtual Service Edge and cluster config with IP address and network parameters
65 ©2019 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION Securing your cloud transformation
• 6
6 Virtual Service Edge Traffic Flow
VMware ESXI
User 1
VM 1
LB1 ZEN1
VIP 1 Webserver
VM 2
LB2 ZEN2
This Setup
User1 makes Contains
aresponds
request a pair
tobackof Virtual
access a webService
serverEdgethe internet and the
LB1Webserver
The and LB2 are configured intoa the request byon
ZEN2 ,and the ZEN 2
VMs, each with 1LB and 1Zen instance packaged in
same is destined
Active/Passive
forwards to the
setup
the response VIP.to
sharing
back Asathe
the LB1 directly.
Virtual
user is Active, it receives the request,
to the VM , all of which are connected to the same
and forwards
IP (VIP) using the request to one of
CARP protocol. In the
thisZENs in the cluster. In this Case
Vswitch
Note: or Port
Zscaler uses group.
the DSR Method to theLoad
of balancing. Hence, theuser Uplink traffic
ZEN2. ZenLB1
scenario, 2 makes the request
is considered as Active, Webserver which the end
response
hence alldoes
intended to notdestined
access.
traffic traversetothrough
the VIPthewillload balancer.
Downlink traffic
land on LB1.
66 ©2019 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION Securing your cloud transformation
Understanding Vzen(Service Edge) in more detail
sc.conf, vzen.conf and vzen_custom.conf
• Vzen has multiple config files that it references.
• /sc/sme/conf/sc.conf
• /sc/sme/conf/vzen.conf
• /sc/sme/conf/vzen_custom.conf 🡪 Optional
• sc.conf has all the generic SME configurations, SMAVD1, SMAVD2 config & CDSC(client) config.
• This file is same across all the Vzen’s
• vzen.conf contains customer specific configurations, as configured in admin portal, apart from cluster
configuration.
Service IP of Vzen Default gw as configured in admin portal
Cluster configuration,.
Helps ensure that
only DPPC of a
specific org is
allowed through
vzen.
Configuration to support ZCC auth
67 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Understanding Vzen(Service Edge) in more detail
sc.conf, vzen.conf and vzen_custom.conf
• Vzen has multiple config files that it references.
• /sc/sme/conf/sc.conf
• /sc/sme/conf/vzen.conf
• /sc/sme/conf/vzen_custom.conf 🡪 Optional
• vzen_custom.conf:This is an optional config, which overrides all the other two config files.
• One of the major use case for this is seen when configuring Vzen in dual arm setup. (Link)
Cluster configuration,.
68 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Understanding Vzen(Service Edge) in more detail
crontab
• Any config changes that are made in portal, are pulled by Vzen.
• These changes are polled by vzen every 5 minutes with help of crontab entry for vzen_mgmt.sh script.
• vzen probes availability of any new build to download using this script.
• vzen probes availability of new security feed tar file, cert bundle update etc using this script.
Cluster configuration,.
69 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Understanding Vzen(Service Edge) in more detail
zscaler.log
• All service restarts are logged in this file.
• vzen can have 1 SMLB plus 1 SME or just be deployed in standalone mode. Number of instances running can be seen by
following command:
• As mentioned earlier, we check for new build every 5 minutes(via crontab), so this file is also updated every 5 minutes .
Logs when there is no build update and only security feed update Logs when there is a build update
PID of SMAVD1
PID of SMAVD2
Cluster configuration,.
PID of SME
71 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Understanding Vzen(Service Edge) in more detail
vzenscripts.log
• All vzen related commands executed by the admin will be logged in this file.
• So, it’s easy to correlate what are the sequence of commands a customer executed, when they ended up in a
certain state.
PID of SMAVD1
PID of SMAVD2
Cluster configuration,.
PID of SME
72 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Troubleshooting Vzen
Master SMLB
PID of SMAVD1
PID of SMAVD2
Cluster configuration,.
Vzen2 is the MASTER in this cluster.
PID of SME
73 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Troubleshooting Vzen
Taking a pcap
• To take a pcap, it must find the interface on which to take a pcap.
• SMLB and SME each have a tap interface which can be used to take pcap.
• First job is to identify which tap interface is owned
by whom.
PID of SMAVD1
PID of SMAVD2
Cluster configuration,.
• This shows that 87411 is SME process and 87949 is SMLB process. So tap1 was opened by SME and tap2 by SMLB.
• Now we can take pcap on SMLB using following command:
PID of SMAVD1
PID of SMAVD2
PID of SME
75 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Troubleshooting Vzen
Vzen troubleshoot connection
• Vzen comes with few commands which can quickly help you identify if there is any connectivity problem.
Cluster configuration,.
PID of SME
76 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Common Issue Seen
Master/Master(Split Brain)
• One of the most common issues we see with Vzen is split brain.
• In this scenario, customers see that SMLB instances are MASTER, thereby causing traffic outage.
• Customer must ensure that they follow this article for configuring clusters.
https://help.zscaler.com/zia/configuring-vzen-clusters
• The Promiscuous mode option must be enabled (i.e., set to Accept) on the vSphere switch (vSwitch). Zscaler load
balancers and ZENs use the Common Address Redundancy Protocol (CARP) to process traffic across multiple ZEN
instances. In order to support this, enabling promiscuous mode on your VZEN interfaces are required.
• The MAC address changes option must be enabled
• The Forged transmits option must be enabled PID of SMAVD1
PID of SMAVD2
• If multiple physical ports exist on the same vSwitch, then the Net.ReversePathFwdCheckPromisc option must be
enabled (i.e., set to 1). If it is not, then multicast traffic will loop back to the host, causing CARP not to function properly,
and "link states coalesced" messages to be sent.
• NOTE: Even after following all these steps, if issue persists, then customers must contact Vmware.
PID of SME
77 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
What must not be done
vzen cleanup
• This wipes the entire config from the vzen as well as the build.
PID of SMAVD1
PID of SMAVD2
PID of SME
78 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Virtual Machines (NSS)
79 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
NSS
Introduction
• NSS uses a virtual machine (VM) to stream traffic logs in real time from the Zscaler Nanolog(SMSM) to customers security
information and event management (SIEM) system, such as Splunk or ArcSight.
PID of SME
80 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
NSS
Introduction
• An organization can deploy the NSS instance either on-premises on an ESX Virtual Machine, as an EC2 Instance on
AWS, or as a Virtual Machine on Azure.
• When an NSS receives the logs from the Nanolog, it decompresses and detokenizes them, applies the configured
filters to exclude unwanted logs, converts the filtered logs to the configured output format so they can be
consumed and parsed by SIEM, then streams the logs to your SIEM over a raw TCP connection.
PID of SMAVD1
PID of SMAVD2
PID of SME
81 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Nanolog and Log Streaming Service Overview
…
Employee
Log Router
NSS FW Syslog
Nanolog Cluster (N+2) Outbound Connection
(6 months, counters)
UI Server
…
Admin UI
Z-Data Lake
©2019 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION Securing your cloud transformation
NSS
Reliable log delivery mechanism
2. NSS to Nanolog: If the connectivity between our cloud and NSS is interrupted, NSS will miss logs that arrived
to the Nanolog cluster during the interruption, and they won’t be delivered to the SIEM. Once the connection
is restored, the NSS one-hour recovery allows the Nanolog to replay logs up to one hour back.
PID of SMAVD1
PID of SMAVD2
PID of SME
83 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
NSS
When to use NSS
• Use Case1
• Zscaler commits to storing 6 months of customer data in our SMSM servers.
• After log retention time period, we delete the older logs.
• If customer require logs to be stored for more duration, they must get NSS and store the logs at their end.
• Use Case2
• In Zscaler Admin UI, customer can search for historical logs. But this tool is not meant for heavy or CPU intensive queries.
• For examples, filtering logs with “URL contains” etc. are CPU intensive actions, which should be done on customer
SIEM.
• Use Case3
• Customers may want custom reports, which are not present in admin UI today.
PID of SME
84 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
NSS
Licenses
• NSS for Web Logs
85 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
NSS
Cloud-To-Cloud Log Streaming (Cloud NSS)
• This allows direct cloud-to-cloud log streaming for all types of ZIA logs into a compatible cloud-
based SIEM solution.
• Rather than deploying, managing, and monitoring on-premise NSS VMs, you can simply configure
an HTTPS API feed that will push logs from the Zscaler cloud service into an HTTPS API-based log
collector on the SIEM.
86 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
NSS
Admin UI configuration.
87 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
NSS
Admin UI configuration.
• Once NSS is added, you can see an SSL certificate against the NSS. This is NSS client cert, which we
must download and import into NSS, when deploying.
88 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
NSS
Admin UI configuration.
• Now that NSS server is created, second step is to add a NSS feed.
Feed Format
89 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Understanding NSS in more detail
Connections made by NSS
• NSS always makes outbound connection to all entities. There is no inbound connection towards NSS from
the service interface.
• sc.conf has all the configurations needed by NSS to get the interfaces up and set up network.
91 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Troubleshooting NSS
nss troubleshoot feeds
• nss troubleshoot feeds -> This command is used to check the health of the feed.
92 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Troubleshooting NSS
nss troubleshoot connection
• nss troubleshoot connection -> This command is used to check the connectivity from NSS to all entities.
Stable Feed1
No connection to
Feed2
93 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Zscaler Auth Bridge(ZAB)
94 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
ZAB(Zscaler Auth Bridge)
Introduction
● The Zscaler Authentication Bridge (ZAB) is a virtual appliance that you can use to provision as well as authenticate users.
● You can use the ZAB to automatically import user information from an Active Directory (AD) or a Lightweight Directory
Access Protocol (LDAP) server to the Zscaler database, without requiring inbound connections to your directory server.
● The ZAB can be used solely as a provisioning tool in conjunction with another authentication mechanism, such as SAML or
Kerberos. Alternatively, it can be used for authentication using LDAP with SSL client certificates.
● ZAB opens a long-living secure outbound tunnel to the Zscaler Central Authority (CA).the ZAB opens a long-living secure
outbound tunnel to the Zscaler Central Authority (CA). So, unlike AD, there is no inbound connection needed to AD server
here.
● It downloads the authentication profile configuration of your organization from the CA and connects to the directory server. It
synchronizes user information from the directory server to the Zscaler cloud on demand or as scheduled.
95 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
ZAB(Zscaler Auth Bridge)
Provisioning
● As can be seen there is no inbound connection from SMCA to AD
server now.
● You have to allow inbound connection from ZAB IP, which is private IP.
96 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
ZAB(Zscaler Auth Bridge)
Authentication
● As shown in the diagram below, the Zscaler
service communicates only with the ZAB
during the authentication process.
97 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
ZAB(Zscaler Auth Bridge)
Configuration for Auth flow.
• You will have to provide the ZAB URL, when configuring ZAB in Admin UI.
• Customers DNS server must resolve this FQDN to the ZAB IP.
98 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
ZAB(Zscaler Auth Bridge)
Configuration for Auth flow.
• With ZAB enabled, customer can configure synchronization frequency to be as small as 2 hours.
• This is not available in AD auth.
99 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
ZAB(Zscaler Auth Bridge)
Configuration for Auth flow.
100 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
ZAB(Zscaler Auth Bridge)
Troubleshooting
101 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
ZAB(Zscaler Auth Bridge)
Troubleshooting
● To check all connectivity on ZAB, you can run zab test-firewall. This will generate a report which can be retrieved &
checked.
102 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
ZAB(Zscaler Auth Bridge)
Troubleshooting
● zab troubleshoot connection 🡪 Will create a report of all connection related information for us to debug.
Connection to AD is down
103 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
ZAB(Zscaler Auth Bridge)
Key points
● zab troubleshoot connection 🡪 Will create a report of all connection related information for us to debug.
Connection to AD is down
104 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Thank You
©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION