Krishna99Main flow
source
event handling
subflow
---------
sub flow
dont have source or event handling
sub flow can be called by main flow
if you drag and drop try block, you can have event handling in subflow
payload
-----------
using setpayload .. data can be passed from main flow to sub flow #[payload]
using setVariable -- variable can be passed from main flow to subflow
#[vars.varName]
Mule expression language MEL => e.g #[payload.id]
in data weave simply => payload.id
For asynchronous
--------------
drag and drop async scope and inside put the subflow
it will create thread
private flow
-------------
it dont have source, but has error handling.
drag and dropping any connector will create private flow automatically
parameters in http endpoint
--------------------
#[attributes.uriParams.key] e.g http://postoffice.org/london/e126tw (inject key as
eg {id}
#[attributes.queryParams.key] http://mylibarary/book?name=java
for REST webservices
-------------
mule palette > http listener and config
URI, http methods
database
------------
database insert --
database config > driver, connection params
in query have as named parameters
in input parameter => map it to payload parameters
e.g
{
"emp_id" : payload.emp_id;
}
select
------------
database select -- ouput is resultset
in transform message
prepare a json and create meadata type (upload json application>json)
and create a mapping between payload (which is available from previous select) and
metadata and output payload
arary of json in response
---------
import metadata > import json
it creates array type in input payload and metadata type
which you can map it to output payload
response
------------
transform message
for only one record : then can do payload[0].emp_id
store env related things in properties file or yaml file
----------
global elements > configuration properties == you can refer the yaml file
also argument can be passed and can be refered as ${env}
similarly property can read as ${key} e.g
${http.port}
secure properties
====================
add secure configuration module to studio
encrypt password:
setup key and algorithm
add anypoint enterprise security plugin to stuido
or use java -jar secure-properties-tools.jar
global elements > secure configuration properties == you can refer the yaml file
error handling
-----------
on error propagate
on error continue
raise error
error handler
validation
----
drag and drop validation from mule palette
then use ${secure::property}
error handling
--------------------
in error handling section > drag error propagate : choose error type > drag
transform message inside that
error.description
in error response > you can set the payload that custom message we created
variables also can be injected ..like status code or headers
error propagate ==> it catches error type defined (specific exception) and throws
error back at the
main source (with custom payload or message)
in error propagate =? you can configure to catch all exceptions too
to call other webservice
------------
we need to drag http request
then configure (host/base path etc)
Jenkins
-----------------
install git plugin
github integraton plugin
1) checkout job .. .create build trigger for git
jenkins poll github for any commit or repo changes and checkout
2) then create a job for build
mvn clean install
3) job to deploy build onto cloud hub
in pom.xml
in maven plugin
<configuration>
<cloudHubdeployment>
<username>mmm</username>
<environment>sandbox</environment>
<workers>1</workers>
<workertype>Micro</workerType>
<applicationName>worldtimezone</applicationname>
<muleVersion>4.1.4</muleversion>
<password>abc</password>
</cloudHubdeployment>
in build step : mvn package deploy -DmuleDeploy
you can see this application deployed in mulesoft cloud hub > runtime manager
after deployment,
for unit test
------------
application in cloudhub
to run postman collection
install nodejs
install npm
npm install -g newman (use npm)
create a new collection in postman with a get request to cloudhub application
use newman run <postman collection .json>
it renders the report in different format
you can create jenkins file to include all automated
===================================
pipelne
{
agent any
stages{
stage('Build application') { steps {bat 'mvn clean install' } }
stage('deploy') {}
}
}
}
now create only one step in jenkins for checkout / build trigger of git and in
script path .. mention jenkinsfile in pipeline section
pipeleine script from scm
regression testing with postman collection - newman
unit testing with munit
to invoke java function
------------------
use java module
write java class function
drag and drop java invoke static
in configuration : privde class name and method name
in input arguments : {
name : attribues.queryParams.name }
to invoke java non static function
-------------
drag and drop new
provide class name and contructor
target variable where instance is created
then drag and drop java invoke
here provide #[vars.instanceName] in instance
classname, method
args
one way SSL
===============
server keystore contains private certificate associated with server
client trustore contains public certificate (server public key) of the server
for handshake between client and server
client sends a request to server
server sends a public key to client
client then use trusttore like CA, Verizon to verify the server public key
certificate
(if dont have CA, use self signed or client trustore)
then client initiates a connection to server by generating a random string
encrypted using server public key
then if server able to decrypt that data, then connection happens betwen client and
server
in two way SSL
-------------
server contains private keystore of the server, trustore (which contains public key
/ certifcate of client)
client contains pirvate keystore of the client, trustore (which contains public key
/ certifcate of server)
can use java keytool to generate keystore (jks)
keep keystore or truststore in resources folder
and refer that in TLS config of HTTPS listener
SFTP on new or updated file
Matcher (for file pattern or timestamp, size etc)
Post processing action like moving to directory, auto delete
Advanced : non repeatable stream
salesforce
--------
put transform message before connector, so that it generates output metadata
automatically
Create operation
query api for contact object
create account.. get security token in email
setup > new connected app
select OAUTH scope
we will get consumerkey , consumer secret
salesforce.com/services/oauth2/token?grant_type=?&client_id=&client_Secret=&
(content-type as url_encoded, POST)
we will get access token as bearer token
then pass that token in authorization header as Bearer <tokename>
authentication
------------------
generate a separate private/public keypair for the Mule server and then add that
publickey to the accounts authorized_keys (/etc/ssh/auth_keys) file on the SSH
server and put the privatekey on the Mule server (create keystore and refer it in
identity file).
design API
----------
open API
APIARY
RAML - interfaces for API implementation, mocked endpoints
eg.
/{ID}:
get:
put:
/flights:
get:
queryParmeters:
destination:
required: false
enum:
- SFO
- LAX
- CLE
Mule flow
=========
Mule event source ---> Mule event processors ---> connector endpoint
Mule palette includes core modules, error handling, transformers, scopes,
components, batch etc
worker is a dedicated instace of mule which runs in a container
proxy
========
create proxy api using api manager (select api from exchange)
API gateway enforce policies on proxy (rate limiting, spike control, security)
mule structure
-------------
Mule Message(Attributes, payload), variables
asyncs
---------------------------
when using flow reference, events are passed synchronusly between flows
but to pass data asynchonously - can be achieved using VM connector
for intra and inter application communication through asynchornous queues
select VM in mule palete under module and select publish consume and drag to canvas
(add queue (transient))
remove private flow reference in main flow
in sub flow, remove http listener and drop VM listener in source
set the queue name
domain project ==> can be used to share global configuration elements between
applications
expose multiple services on same port, call flows in other applicatios etc
global
---------
encapsulate all global elements in a separate configuration file (global.xml)
new Mule configuration file(global.xml)
move the existing global elements to the global.xml
configure application properties in a properties file or yaml file
use it in connector properties
maven
------
groupid
aretefact id
version
dependency
plugin
structure
-------
src/main/mule
src/main/java
src/main/resources
src/test/munit
define metadata < manage metadata types
----------------
for output structures required transformations
metadata editor can be invoked using
set Metadata on input panel
define metadata on output panel of transform message
under src/main/resources > application-types.xml
use Webservice Consumer config for SOAP
in http listener > advanced - allowed methods : we can set to GET/POST
add an API from exchange to anypoint studio project
=================================
add dependencies
---------------
add modules from exchange
RAML - API specification RAML 1.0
----------
http methods
request payload
response payload
content type
payload validation
URI -> query parameters, template or uri parameters
description
apiname
baseuri
resources
oauth
e.g
/add-employee:
post:
body:
application/json
example:
{
}
responses:
200:
body:
application/json:
example:
{
}
500:
get:
queryParameters:
empId:
required:true
type:integer
example:100
you can do mock service and give to product owner and business before development
make all corrections in the design phase itself
best practices
==============
global configuration file for all configurations (global elements)
global error handler
then flow for each resource is defined in separate configuration files under
implementations folder
then flow reference for them is refered in main flow
under resources folder > create yaml or properties file for each environment or
cofiguration properties.
all files placed in src/main/resources will be avialable to mule esb application
context
under resources (yaml, wsdl, logger, keystore, api specification) [log with splunk
or elastic search - centralized logging]
AS you add any connectors from modules, a module reference will be added into the
project. a dependency will be added in pom.xml
src/main/mule
src/main/java
src/main/resources
src/test/munit
pom.xml
cloud hub deployment
====================
export from anypoint studio to jar
then in anypoint plaform > runtime manager > deploy application, deploy that
second option (recommended)
-------
pubish to exchange from anypoint studio
then from runtim manager > import from exchange in
or
deploy to cloudhub diectly from anypoint studio
it opens runtime manager(need to specify application name, worker size, no of
workers, runtime version 4.2.0)
API management
-------------
steps:
1)publish API specification to exchange from design center of any point platform
2)import to API manager :: >> API Administration > get from Exchange. you can
choose basic endpoint or endpoint with proxy. also speficy implementation url
Then take API id and go to global configuration > auto discovery and specify there
and flow name
deploy it again to runtime manager
APP ID (auto discovery) is the link between the instance running in API manager
with policies and application running in Runtime manager.
Go to access managment > add busines group : take client id and client secret ...
copy that to API Manager window of Any point studio
and deploy the app to cloudhub from studio
configure business group client id and secret
enable auto discovery
apply policies
(client id enforcement (create client application in exchange and get clientid and
client secret and pass it in headers while consuming api), oauth 2, basic auth,
header injection etc, rate limiting)
API fragments
---------
traits
security schemes
data types
examples
API spec defines API
fragment is a resuable logic that can be defined
trait - resuable code ..like x-transaction-id ..or what ever client needs to pass
it in headers
security scheme - like oauth
move all these into separate files and inject them in RAML API spec for
transaction headers, security scheme, base uri parameters, query parameters, data
type, payload etc
!include <.json>
traits:
responsemessage
then using is keywords ..you can refer the trait
OAUTH
======
when we register our application with oauth provider, they give client id, client
secret, grant type as client credentials
when we consume api, we need to request for token by passing clientid and client
secret and grant type, it generates a access token
then after obtaining access token, pass this token in authorizaton header as bearer
in API administration> while applying API policy, select OAUTH2.0 and then give
validateToken url for access token
proxy
--------
in api administration .. select either basic endpoint or endpoint with proxy
scatter gather
------
scatter invoking different requests in parallel
gather - gther different responses in payload array
batch job
----------
batch job has batch step
then oncomplete event
we can specify block size... it splits records in to batch
add batch agreegator, we can add actions (or log) what to do after each batch is
completed.
file
-------
The File, FTP, and SFTP connectors provide a listener (called On New or Updated
file in Studio and Design Center) that polls a directory for files that have been
created or updated.
Set the autoDelete parameter to true. This setting deletes each file after it has
been processed so that all files found in the next poll are new.
Use the watermark parameter to only pick files that have been created or updated
after the last poll was executed.
batch processing of 1M records
------------------
listener or scheduler to trigger the process(cron expression)
invoke java static (method to read the file in chunks and create file in chunks in
output dir)
lisenter for (new or updated file)
pick the chunked file
batch job > batch step -> In try > transform like csv to json -> with DB insert
in errro handling -> on error continue -> faulted folder
another design
-------
using mule4 non repeatable stream and get payload as iterator. chunks of data are
streamed to batch processing
inside batch job, we can use dataweave to transform csv and write it to db etc
Instead of a list of elements that you receive with a fixed-size batch commit, the
streaming functionality – an iterator – ensures that you receive all the records in
a batch job without running out of memory. By combining streaming batch commit and
DataMapper streaming in a flow, you can transform large datasets in one single
operation and one single write to disk.
security
---------
<secure-property-placeholder:config key="${prod.key}" location="test.$
{env}.properties"/>
When you add data to the properties file, Mule gives you the option to encrypt the
data. You then choose an encryption algorithm (of the 19 available), and enter an
encryption key. That encryption key is the only way to decrypt the properties in
the properties file. In other words, the encryption key is the only thing that
unlocks the Credentials Vault.
Next, create a global Secure Property Placeholder element which locates the
Credentials Vault (that is, the properties file), and retrieve encrypted
properties. However, the Secure Property Placeholder can only access the
Credentials Vault (that is, to decrypt the data) if it has the key.
Therefore, you need to configure the Secure Property Placeholder to use the key
that Mule collects from the user at runtime (see code below). In this context, the
key to decrypt the properties becomes a runtime password.
mule maven plugin 2.3 to studio
it generates a pom file
muledeploy
muleundeploy
need to add configuration for cloudhub deployment
<cloudHubDeployment>
<uri>https://anypoint.mulesoft.com</uri>
<muleVersion>${mule.version}</muleVersion>
<username>${username}</username>
<password>${password}</password>
<applicationName>${cloudhub.application.name}</applicationName>
<environment>${environment}</environment>
<properties>
<key>value</key>
</properties>
</cloudHubDeployment>
deployment
-------
Encrypted credentials are available in all platform deployments: CloudHub, Runtime
Fabric, and Runtime Manager. To use encrypted credentials when deploying, you need
to set up your Maven master encrypted password and your settings-security.xml file.
Create a master password for your Maven configuration.
$ mvn --encrypt-master-password <yourMasterPassword>
Maven returns your master password encrypted:
{l9vZ2uM5SdgHy+H12z4pX7LEOZn3Kbnqmt3kIquLjnQ=}
Create a settings-security.xml file in your ~/.m2 repository, and add your
encrypted master password.
<settingsSecurity>
<master>{l9vZ2uM5SdgHy+H12z4pX7LEOZn3Kbnqmt3kIquLjnQ=}</master>
</settingsSecurity>
Encrypt your Anypoint platform password:
$ mvn --encrypt-password <yourAnypointPlatformPassword>
Maven returns your Anypoint platform password encrypted:
{HTWFGH5BG9QmvJ1B=}
Add your encrypted Anypoint Platform password to your settings.xml file in the
<server> element:
<settings>
...
<servers>
...
<server>
<id>my.anypoint.credentials</id>
<username>my.anypoint.username</username>
<password>{HTWFGH5BG9QmvJ1B=}</password>
</server>
...
</servers>
...
</settings>
In your configuration deployment, reference the credentials injecting the server id
configured in your settings.xml file. Below is an example of a CloudHub Deployment
using encrypted credentials:
<plugin>
...
<configuration>
<cloudHubDeployment>
<uri>https://anypoint.mulesoft.com</uri>
<muleVersion>${mule.version}</muleVersion>
<server>my.anypoint.credentials</server>
<applicationName>${cloudhub.application.name}</applicationName>
<environment>${environment}</environment>
<properties>
<key>value</key>
</properties>
</cloudHubDeployment>
</configuration>
</plugin>
throttling
--------
no of requests per min
configure that in SLA tier of API administration
then add Rate limiting in API policy
apache CXF module to consume SOAP webservice
Aggregating the Payload (splitter and collection aggregator)
------------------------
When the splitter splits a message, it adds three new outbound variables into each
of the output fragments. These three variables are later used by the Aggregator to
reassemble the message:
MULE_CORRELATION_GROUP_SIZE: number of fragments into which the original message
was split.
MULE_CORRELATION_SEQUENCE: position of a fragment within the group.
MULE_CORRELATION_ID: single ID for entire group (all output fragments of the same
original message share the same value).
File -> splitter -> VM
VM -> resequencer -> collection aggregator -> logger
both VMs share same Queue
scatter and gather
-------------
The routing message processor Scatter-Gather sends a request message to multiple
targets concurrently. It collects the responses from all routes, and aggregates
them into a single message.
unit test
--
org.mule.tck.AbstractMuleTestCase
questions
===========
mule FTP - authentication
how do u connect to salesforce cloud
backout queue and dead letter queue
private flow and subflow
ssh-keygen to generate both public and private keys -- public key will be created
in keystore under resources which will be imported in identity file in
connector config
PGP for encryption and decryption
------------
Global error handler
error handling > reference excepton strategy
Once we have imported the jar as maven dependency, we need to specify Mule file
name present in the JAR in our Mule Project.
Go to Global Elements > Create > Expand Global Configuration > Import
In Error handling part, “On Error Continue” is checking for the retry count if it
has reached to its max or not. Inside error flow of “On Error Continue” retry count
value is getting incremented and after some seconds of sleep; flow reference will
again call HTTPFlow
Drag and Drop the JSON Validate Schema from Mule Palette to validate the input
payload.
In DataWeave 2.0 functions are categorized into different modules.
Core (dw::Core)
Arrays (dw::core::Arrays)
Binaries (dw::core::Binaries)
Encryption (dw::Crypto)
Diff (dw::util::Diff)
Objects (dw::core::Objects)
Runtime (dw::Runtime)
Strings (dw::core::Strings)
System (dw::System)
URL (dw::core::URL)
create Munit suite by right clicking API starter
each munit flow will have setpayload in behaviour, execution section for request,
validation with assertion
To Mock a connector, we need to place “Mock When” in Behavior section. And define
its configuration.
You can set the processor attribute to define the processor to mock with the
connector namespace and operation; and the with-attribute element to define the
connector’s attribute name and value so that mule can identify which connector is
to be mocked. “Then-return” you can define the message that is to be returned by
the connector.
create JavaException class
register in mule.xml as global function
validation done in dataweave and call function to raise exception
resourceTypes:
collection: item: for single item in collection
usage: Use this resourceType to represent any collection of items
description: A collection of <<resourcePathName>>
get:
description: Get all <<resourcePathName>>, optionally filtered
responses:
200:
body:
application/json:
type: <<typeName>>[]
/foos:
type: { collection / item: { "typeName": "Foo" } }
securitySchemes:
customTokenSecurity : !include securitySchemes/security.raml
apply to APIs
securedby: customTokenSecurity
database polling
----------------
create poll scope
drop datbase select inside poll scope
put the flow processing strategy to synchronous
create flow variable in watermark in poll scope and use selector expression e.g
#[payload.id]
then in database > query ..chang to where ID > #[flowVars.varname]
so that it picks up only new records
correlation/aggregation pattern
-----------
flattern reduce
JMS
flow processing strategy
https://www.youtube.com/watch?v=i6hX8WeVtD4
traits vs resource types
