Modern APIs Old IT operating model (Point to point integrations or tightly coupled) Resuable assests, ease of consumption, performance, scalability API-led connectivity approach (Experience APIs, Process APIs, System APIs) Center for Enablement(c4e) PaaS as “Anything that is not a business application or systems infrastructure is PaaS” API - could be referring to API Designer - contains File browser, Editor, Shelf, API Console (visual representation of specification, available in API designer, API portals and in studio via APIKit) Anypoint Studio - Eclipse based IDE using Spring framework used to develop, deploy, test and analyse Components - Package explorer, Canvas, Console, Mule palette. Global Elements, Message Flow, Global XML configuration Mule palette - Core modules, Other/Featured modules/connectors, modules or connectors from exchange Core module - elements are 7 Batch (3) - Batch Aggregator, Batch Job, Batch Step Components (6) - Flow Refs, Logger, Transform message, Parser Template, Invalidate key, Invalidate cache Endpoints (1) - Scheduler Error Handling (4) - Error handler, On error continue, On error propogate, Raise Error Flow Control (4) - Choice, First Successful, Round Robin, Scatter-Gather Scopes (7) - Async, Cache, Flow, Subflow, Foreach, Try, Until Successful Transformers (3) - Set payload, Set variable, Remove Variable Other/Featured modules are 20 APIKit Database - Bulk Insert, Bulk Delete, Bulk Update, Insert, Delete, Update, Select, Execute Script, Execute DDL, On Table Row Objectstore - Clear, Contains, Remove, Retrieve, Retrieve all, Retrieve all keys, Store. Java - Invoke, Invoke static, New, Validate Type File (not among feature) - Read, Write, Delete, Copy, Rename, Move, List, Create directory, On New or Updated File Email FTP - Same as File SFTP - Same as File HTTP JMS - Same as VM + "Ack" and "Recover Session" VM - Consume, Publish, Publish-Consume, Listener (both inter and intra app) Validation Scripting - Execute OAuth - Retrieve access token, Retrieve expires in, Retrieve refresh token (Resource owner id) OAuth 2.0 Provider - Create client, Delete client, Revoke token, Validate token Salesforce - create, create batch, create bulk, query Sockets -send, receive, send and receive Spring Web service consumer - Consume Service Now - Invoke (soap operation by resource owner id), unathorised (invalidate all tokens of resource id) XML SAP - Asynchronous RFC, Synchronous RFC, Retieve IDOC, Send IDOC, Document Listener, Confirm Transaction, Function Listener AS400 - Read data queue, Write data Queue, Program Call Processor, Command Call, Read data queue processor (program Library, program name, procedure name, Data queue) modules or connectors from exchange (secure properties tools, property placeholder editor, cloudhub notifications using alerts) Flow - Event Source, Event Processors, Error Handling Dataweave (As a functional programming language (Used for code transformation and Data Enrichment)) 1. Type systems (Applied on variables and Functions(all about constraints) Categories: Simple - Atomic(String, Boolean, Number, Regex, Null, Temporal(Date, Time, DateTime, LocalTime, LocalDateTime, Period)) Composite (Array, Object (Closed, Open) defined by type or {} or {| |}, Functions (T) -> T) Complex (Any, Nothing, Union | on simple types, Intersection & on objects, Literal (string, number, boolean), function overloading Type Paramters (Binding Type parameters)fun head(elements: Array): om = elements[1] Auto/Default coercion 2. Data Selectors (Single value, Multi Value, Descendent, key-value pair, index, range, Assert Present, Namespace, XML Attribute, Key Present, Filter 3. Dataweave Variables 4. Reserved Variables (and, or, as, is, async, case, default, do, else, if, enum, false, true, for, import, input, output, ns, type, null, private, throw, unless, var, fun, yield, using) 5. Script structure Header import, var, fun, type, ns output, input Supported Data Formats (reader properties, writer properties (skipNullOn, separator, header, deferred, escape)) application/{csv,json,java,xml,yaml,dw,flatfile,avro,octet-stream,xlsx,x-www-form-urlencoded}, multipart/form-data, text/plain, text/x-java-properties Body 6. Operators 5 Mathematical (+, -, *, /) # + ,- operates on arrays, objects, dates Equality and Relational(<,>,<=,>=,==,~=) Logical (not, !, and, or) Prepend >>, Append <<, +, - on Arrays Scope and Flow Control (do, using, if else, else if) 7. Core (dw:Core) Functions (++, --, filter, filterObject, distinctBy, orderBy, groupBy, joinBy, splitBy, sizeOf, map, mapObject, pluck, reduce, replace, with, contains, flatMap, flatten, match, matches, read, readUrl, now, keysOf, valuesOf, namesOf, entriesOf, scan, trim, to, log, mod, find, floor, ceil, abs, avg, round, isInteger, isEmpty, isBlank, isEven, isOdd, isDecimal, isLeapYear, pow, sqrt, startsWith, min, max, minBy, maxBy, lower, upper, uuid, random, randomInt, zip, unzip) Types (Any, Array, Object, CDATA, Binary, Boolean, Date, DateTime, Key, Enum, Null, Number, Range, Regex, Period, LocalDateTime, Time, TimeZone, Type, Uri, Iterator 8. Modules Arrays , Binaries,Encryption Diff, Objects, Runtime, Strings, System, URL Inline Dataweave, script parameterisation, commenting Mule Runtime Functions (p, lookup, causedBy<#[error Mule::causedBy 'HTTP:UNAUTHORIZED']>)(Mule namespace) Predfined Mule Variables (payload, attributes, vars, app, flow, correlationId, error, message, server, mule) Mule Events (Mule Message (Payload, Atributes), Variables) Configuring properties Property placeholders, Global properties, Properties files, Supported Files, System properties, File Properties, Environmental variables Dynamic properties Property Heirarchy (Deployment props > System props > Environment props > application props (configuration props, secured conf props) > Global properties import Mule Message Data Types (Mutable, Immutable), Data Streams (Repeatable, Non Repeatable) File Stored Repeatable greater than writes to disk, InMemory Repeatable greater than fails File Stored Repeatable (iterable) greater than writes to disk InMemory Repeatable (iterable) Transactions (action: EMPTY, ALWAYS BEGIN, BEGIN_OR_JOIN, INDEFFERENT(default)),(types: local((default), XA),(eligiblity: VM, DB, JMS) Shared resources and Domains, exported resources and packages Timeouts (Connection timeout, Response timeout, Request Timeout, Max Idle time) Reconnection Strategies (none, standard, forever) Redelivery policy(max retries, objectstore counter, secure hash), and retry mechanisms Mule Errors Error Selectors < #[error.>,childErrors>]> Error Types ANY UNKNWON, TRANSFORMATION, ROUTING, CONNECTIVITY, EXPRESION, SECURITY , VALIDATION CRITICAL Error Handlers , onErrorPropogate, GlobalErrorHandler> Error Mappings map to custom or common errortype> Mule Data Processing Exceptions Mule SDK Classes: (Extension Class, Operations Class, Supporting Classes), dependencies in pom Features: apache logger, Scheduled task executor, RouterCallBack Method, Routes and Paramter declaration, flow stop and flow start using mule registry, concurrent wait and input streams, hashmaps, icons Connector Configuration HTTP Listener: (protocol, host, port, tls: ) (conf: mimeType, redelivery, responses>, advanced) on step level, Scheduling strategy, job instance ID, batch block size, max concurrency, target) Batch Step (Accept Expression, Accept Policy) Batch Aggregator (Streaming preserve, mime type preserve, Aggregator size) Batch job result, status, id #[payload.*] (loadedRecords, failedRecords, processedRecords, successfulRecords, batchJobInstanceId, timeElapsedinMills, Exceptions) Batch rules: Complete parallel processing, no batch step will wait, no batch step components will wait, no batch aggregator will wait As soon as a record completes with its previous component it moves on unpredictable processing and logging. Tracking and sequencing is not possible. A batch is similar to async. Batch job phases: load and dispatch, process, oncomplete variables and payload only pass between Batch Steps, not between steps and aggregator, or aggregator to steps or Batch jobs to flow. SAP conf: IDOC Library, Simple Connection Provider, JCO Library, Username, Password, SAP System number, SAP Client ID, Host - Phases - Goals (aka Plugin Goals) Usage - mvn , mvn : or :[:]:. Brief points - 1. Build lifecycles consists of Build Phases. Most maven phases has a default goal bindings. 2. Build phases consists of Build Goals or Plugin Goals 3. Plugins are a collection of Goals also called MOJO (Maven Old Java Object). (Analogy : Plugin is a class and goals are methods within the class) 4. When you run a phase, all the phases upto that phase and along with that phase will also run. 5. When you run a phase, all goals attached/binded to that phase runs. 6. A phase can have multiple goals (zero or more goals). Phase with no goals will not run. 7. A goal can be binded to multiple phases. A goal with no phases binding can be directly invoked. 8. Phase are meant to run and Goals are meant to be executed. 9. A plugin can have more goals specified which can be called upon individually. 10. Default bindings for a phase can vary and depend upon the type of packaging specified. pom (modelVersion, groupId, artifactId, version, packaging, name, properties, organization, inceptionYear, licenses, developers, distrubtionMangement,scm, dependencies, build(resources>, plugins, executions>>>) repositories-*, pluginRepositories-* profiles-* List of mule classifiers: mule-plugin, scope: (test, provided) packaging: mule-extension, mule-application, pom, jar, maven-plugin Log4j2 Logging Levels (ALL < TRACE < DEBUG < INFO < WARN < ERROR < FATAL < OFF ) Components of Log4j (Appenders, Configuration properties, pattern layout<: %r [%t] %-5p %c{2} %x - %m%n><="%d [%t] ${sys:hostName} %-5p %c - %m%n>, Policies, filters, Aysnc Root-Appender Ref) log destinations (local machine, Database, Cloud, Sockets, Console, Cloudwatch, AWS Elastic Search, Amazon S3, kibana, MySQL) OAuth 2.0 (Authorization Server, Resource Server, Client, Resource Owner) Grant Types: Authorization Code, Implicit,Username & password, Client Credential Oauth dance: 1. Client Request access token to OAuth server 2. Authenicate client at OAuth server 3. Return access token to client from OAuth server 4. Client requests resource with access token 5. Resource server validates access token with OAuth server 6. Validated access token response to Resource server 7. Resource server returns the resource Archetypes project-name src/main/resources archetype-resources src/main/.. src/test/.. pom.xml META-INF/maven/archetype-metadata.xml archetype-descriptor fileSets and filters src/test pom.xml (archetype-packing) Project Structure and Features global.xml secure properties tools (filename, key, file level encryption, encrypt alogorithm, encyrpt mode) configuration properties (file name) Auto Discovery (API Id, flowName
) import (like common_error_handle.xml taken from project libraries) muleDeploy.groovy (appName, env list, apiUri, workerForEnv, autoDiscovery conf, cloudhubApp Properties, azureAD policies mule-artifact.json (minMuleVersion, secureProperties) gdsl, intelliJ Features Azure devops (Pipelines, git, boards, pull requests) Azure AD (app registrations, key-vault, JWT) app regis: (application (client) ID, tentant ID, object ID, Application ID URI) Certificates & Secrets (Certificates, Client Secrets) for Basic Functionality of API Owners (who can edit all the above) MANFIEST XML Not every PR merge to the master goes to to every stage Pipelines (Stages: Build, Dev, QA, Prod) Build: Initialize job, Pre-job Downloads, Download secrets from key-vault, Checkout api, Maven repo creds/settings.xml, preserve pom.xml for caching, Cache Maven Packages, Fetch Maven Dependencies, Set Version Number, Run unit tests, Build and deploy, Copy Files to, Publish Artifacts to drop, Post-jobs, Finalize job Deploy: Initialize job, Pre-job, Deploy to Cloudhub, Post-job, Finalise job JIRA/Boards (Boards, Backlogs, Sprints, Calendar) Connector driven connectivity among APIs Custom Connectors like Circuit Breakers and Custom Loggers Integrations taken up SAP (IDOCs and RFCs) to LCT, Snowflake to Facebook, Snowflake to LCT, OTM(Transmission XMl) to LCT, OTM to WMS Control-M Best Practices Following Widely used Integration Patterns (Migration, Broadcast, Bidirectional sync, Correlation, Aggregation) Clean project structure (Separate XMls for Global Elements, Error Handling, Implementation Logic. Different Packages for java, apis, wsdls, examples, test cases etc) Naming conventions and Identations Use of Conf Props File and Overriding with defaults Externalising the scripts rather inline Munits coverage standards Message Brokers -> Kafka, ActiveMQ, AWS SNS, AWS SQS, RabbitMQ, Azure Service Bus Message Business Value: Volatile, Durable, Replayable; Semantics: Command, Query, Event Queues for P2P, Topics for 1toMany Kafka: Higher Throughput, only topics, limited record size, TCP, Permanently stored, Offset, Commit log Cluster: group of brokers(server containers); Broker: group of topics; Topic: many partition, Topic duplication, replication factor, Leader Conf: Poll timeout, Acknowledgment mode (Manual, Auto etc), Amount of parallel consumers General Conf: bootstrap url, Group ID (aka Consumer GroupID), Topics, SASL/PLAIN(username, password) Advanced: Isolation level (READ_COMMITED, READ_UNCOMMITED), Auto offset reset (LATEST, ERROR, EARLIER), Retry backoff timeout, Default send buffer size Default receive buffer size, Request timeout, Default record limit, FETCH Conf: Default fetch minimum size attributes *consumerCommitKey topic partition offset creationTimestamp operation: commit, message listener, publish, consume, batch message, seek Subflow, Flow, Privateflow Java 8 Features Default and static methods in interface Functional interfaces Lambda expressions Stream API, Date API