Making a barn door
dimensions:
54 w”
83 h”
1″ x 6″ x 8′ boards – 10
1″ x 6″ x 10’board – 1
1″ x 3″ x 8′ boards – 2
1″ x 3″ x 10′ board – 1
one package of 1 1/4″ trim head screws
wood glue
wood filer
sending block
stain honey 272
polyurethane
Vanilla LSTM - Memory cells of a single LSTM layer used in simple network structure Stacked LSTM - Deep networks as LSTM layers are stacked one on the top of another CNN LSTM - A convolutional NN used to learn features in spatial input like images and to generate image responses. Encoder-Decoder LSTM - Separate networks of LSTM's that encode input and decode output sequences Bidirectional LSTM - Input sequences are presented and learned both forward and backward Generative LSTM - LSTM's learn the structure relationship in input sequences so well that they can generate new plausible output sequences
Sequence prediction is a type of supervised learning problem. The sequence imposes an order on the observations that must be preserved when training models and making predictions. We will examine four different types of sequence prediction problems:
1. Sequence Prediction.
2. Sequence Classification.
3. Sequence Generation.
4. Sequence-to-Sequence Prediction.
Often we deal with sets in applied machine learning such as a train or test set of samples. Each sample in the set can be thought of as an observation from the domain. In a set, the order of the observations is not important. A sequence is different. The sequence imposes an explicit order on the observations. The order is important. It must be respected in the formulation of prediction problems that use the sequence data as input or output for the model.
Sequence Prediction
Sequence prediction is sequence learning.
Examples:
Weather Forecasting – Given a sequence of observations about the weather over time,
predict the expected weather tomorrow.
Stock Market Prediction – Given a sequence of movements of a security over time,
predict the next movement of the security.
Product Recommendation – Given a sequence of past purchases for a customer, predict
the next purchase for a customer.
Appendix:
API Documentation and White papers:
https://arxiv.org/
https://scholar.google.com/
Sequence classification involves predicting a clkass label for a given input sequence, e.a.
1,2,3,4,5
good
Examples:
DNA Sequence Classification. Given a DNA sequence of A, C, G, and T values,
predict whether the sequence is for a coding or non-coding region.
Anomaly Detection. Given a sequence of observations, predict whether the sequence is
anomalous or not.
Sentiment Analysis. Given a sequence of text such as a review or a tweet, predict
whether the sentiment of the text is positive or negative.
Sequence Generation
Sequence generation involves generating a new output sequence that has same general characteristics as other sequence in the corpus
1,3,5 and 7,9,11 has output 3,5,7
RNN can be trained for sequence generation by processing real data sequences one step at a time and predicting what comes next. Assuming the predictions are probabilistic, novel sequences can be generated from a trained network by iteratively sampling from the network’s output distribution, then feeding in the sample as input at the next step. In other words by making the network treatits inventions as if they were real, much like a person dreaming.
Examples:
Text Generation. Given a corpus of text, generate new sentences or paragraphs of text that read they could have been drawn from the corpus.
Handwriting Prediction. Given a corpus of handwriting examples, generate handwrit-
ing for new phrases that has the properties of handwriting in the corpus.
Sequence Prediction Problems
Music Generation. Given a corpus of examples of music, generate new musical pieces
that have the properties of the corpus.
Sequence generation may also refer to the generation of a sequence given a single observation as input. An example is the automatic textual description of images.
Image Caption Generation. Given an image as input, generate a sequence of words
that describe an image. Something like “Sasha riding motorcycle image” and output sequence is “man riding a motorcycle”
Sequence to Sequence Prediction
Predict output sequence from given input sequence.
1,2,3,4,5
6,7,8,9,10
Despite their flexibility and power, [deep neural networks can only be applied to
problems whose inputs and targets can be sensibly encoded with vectors of fixed
dimensionality. It is a significant limitation, since many important problems are best
expressed with sequences whose lengths are not known a-priori. For example, speech
recognition and machine translation are sequential problems. Likewise, question
answering can also be seen as mapping a sequence of words representing the question to a sequence of words representing the answer.
Sequence-to-sequence prediction is a subtle but challenging extension of sequence prediction, where, rather than predicting a single next value in the sequence, a new sequence is predicted that may or may not have the same length or be of the same time as the input sequence. This type of problem has recently seen a lot of study in the area of automatic text translation (e.g. translating English to French) and may be referred to by the abbreviation seq2seq. seq2seq learning, at its core, uses recurrent neural networks to map variable-length input sequences to variable-length output sequences. While relatively new, the seq2seq approach has achieved state-of-the-art results in […] machine translation.
If the input and output sequences are a time series, then the problem may be referred to as multi-step time series forecasting. Some examples of sequence-to-sequence problems include:
Multi-Step Time Series Forecasting. Given a time series of observations, predict a
sequence of observations for a range of future time steps.
Text Summarization. Given a document of text, predict a shorter sequence of text that
describes the salient parts of the source document.
Program Execution. Given the textual description program or mathematical equation
predict the sequence of characters that describes the correct output.
Limitations of Multilayer Perceptrons
Classical neural networks called Multilayer Perceptrons, or MLPs for short, can be applied to sequence prediction problems. MLPs approximate a mapping function from input variables to output variables. This general capability is valuable for sequence prediction problems (notably time series forecasting) for a number of reasons.
Robust to Noise. Neural networks are robust to noise in input data and in the mapping
function and can even support learning and prediction in the presence of missing values.
Nonlinear. Neural networks do not make strong assumptions about the mapping function and readily learn linear and nonlinear relationships.
More specifically, MLPs can be configured to support an arbitrary defined but fixed number of inputs and outputs in the mapping function. This means that:
Multivariate Inputs. An arbitrary number of input features can be specified, providing
direct support for multivariate prediction.
Multi-Step Outputs. An arbitrary number of output values can be specified, providing
direct support for multi-step and even multivariate prediction. This capability overcomes the limitations of using classical linear methods (think tools like
ARIMA for time series forecasting). For these capabilities alone, feedforward neural networks are widely used for time series forecasting. One important contribution of neural networks – namely their elegant ability to approximate arbitrary nonlinear functions. This property is of high value in time series processing and promises more powerful applications, especially in the subfeld of forecasting …
MLP’s have five critical limitations:
Stateless MLPs learn a fixed function approximation. Any outputs that are conditional on the context of the input sequence must be generalized and frozen into the network weights.
Unaware of Temporal Structure Time steps are modeled as input features, meaning
that network has no explicit handling or understanding of the temporal structure or order between observations.
Messy Scaling For problems that require modeling multiple parallel input sequences,
the number of input features increases as a factor of the size of the sliding window without any explicit separation of time steps of series.
Fixed Sized Inputs The size of the sliding window is fixed and must be imposed on all
inputs to the network.
Fixed Sized Outputs The size of the output is also fixed and any outputs that do not
conform must be forced.
The computational unit of the LSTM network is called the memory cell, memory block, or just cell for short. The term neuron as the computational unit is so ingrained when describing MLPs that it too is often used to refer to the LSTM memory cell. LSTM cells are comprised of weights and gates.
The Long Short Term Memory architecture was motivated by an analysis of error
flow in existing RNNs which found that long time lags were inaccessible to existing
architectures, because backpropagated error either blows up or decays exponentially.
An LSTM layer consists of a set of recurrently connected blocks, known as memory
blocks. These blocks can be thought of as a differentiable version of the memory
chips in a digital computer. Each one contains one or more recurrently connected
memory cells and three multiplicative units – the input, output and forget gates –
that provide continuous analogues of write, read and reset operations for the cells.
… The net can only interact with the cells via the gates.
A memory cell has weight parameters for the input, output, as well as an internal state that is built up through exposure to input time steps.
Input Weights. Used to weight input for the current time step.
Output Weights. Used to weight the output from the last time step.
Internal State. Internal state used in the calculation of the output for this time step.
LSTM Gates
The key to the memory cell are the gates. These too are weighted functions that further govern the information flow in the cell. There are three gates:
Forget Gate: Decides what information to discard from the cell.
Input Gate: Decides which values from the input to update the memory state.
Output Gate: Decides what to output based on input and the memory of the cell.
We can summarize the 3 key benefits of LSTMs as:
Overcomes the technical problems of training an RNN, namely vanishing and exploding gradients.
Possesses memory to overcome the issues of long-term temporal dependency with input sequences.
Process input sequences and output sequences time step by time step, allowing variable length inputs and outputs.
Applications of LSTMs
Automatic Image Caption Generation
Automatic Translation of Text
Automatic Handwriting Generation
LSTMs are very impressive. The design of the network overcomes the technical challenges of RNNs to deliver on the promise of sequence prediction with neural networks. The applications of LSTMs achieve impressive results on a range of complex sequence prediction problems. But LSTMs may not be ideal for all sequence prediction problems. For example, in time series forecasting, often the information relevant for making a forecast is within a small window of past observations. Often an MLP with a window or a linear model may be a less complex and more suitable model.
MobiusR provides science around data and technology and expertise to help organizations turn Big-Data into Big results. We are founded on the belief that data collected internally and otherwise is to a company a most valuable economic asset and it should be treated like that. Mining and learning from that data is value for creation. Using a unique blend of business acumen, big-data integration, ML, deep learning and simple run-time decision support tools MobiusR could provide your company with incredible insights into data that you have or interact with. Our experience is broad and interwoven with history of specialization across multi disciplines with expertise in Risk Management, IOT, Supply-Chain Management, Service Management, Web Analytics, Bio Informatics, Finanance and Academia and Government.
MobiusR technologists every day work on the following subject areas, with clients or otherwise:
Apply Big Data and Data Science applications to every day business models.
Discover by manipulating data inputs and outputs patterns that are of higher interests for business.
Apply and solve previously unsolved problems using ML and DL techniques and correlate and apply those findings within a a whole data eco system.
Build on top of these systems Support Tools to allow business makers to anticipate issues, restructure company resources and improve business outcomes.
MobiusR idea of Big Data systems is micro and nano software architecture that is embedded, run time driven, easily configurable, run time scalable and highly modular providing R&D type of technology stack with production reediness.
free and relatively easy toolchain for processing aerial drone imagery. Typical drones use simple point-and-shoot cameras, so the images from drones, while from a different perspective, are similar to any pictures taken from point-and-shoot cameras, i.e. non-metric imagery. OpenDroneMap turns those simple images into three dimensional geographic data that can be used in combination with other geographic datasets.
Not a bad show to watch to learn about drones as business
her is how you can do mapping areal mode and then carry this water drone to river or lake and get the sonar to work … (or fish finder)
Above drone is robust and tractor like machine
Now you could carry this one with another airborne drone like this one:
The Yeair is a quadcopter that is in the planning stages and is expected to be released to the public very soon. Although it costs a little more than a thousand dollars, it is a strong model that works with a gas engine but is also paired with a small battery cell that will help to power up the unit.
This particular quadcopter can go up to 60 mph in speed and can carry about twelve pounds on average. It can also stay in the air for about an hour on average before it has to be refueled. The battery that comes with the Yeair will not have to be recharged although it will at least keep the emissions from the model from being more of an issue than necessary.
The Yeair uses a two-stroke engine and will not wear out easily like what might come about when a conventional drone is used. Still, the quadcopter is a point that helps with making it easier for the model to be used the right way.
you can have int64 and float64…
dtype – data type object
The default data type is floating point
import numpy as np
a = np.array([1,2,3])
a.dtype
dtype('int64')
b = np.array([1.,2.,3.])
b.dtype
dtype('float64')
You can explicitly specify which data-type you want:
c = np.array([1, 2, 3], dtype=float)
c.dtype
dtype('float64')
int32int64uint32uint64Someone more eloquent than me said: in NumPy there are two different data types for dealing with rows and columns of numbers. Be careful of this because they look similar, but simple mathematical operations such as multiply on the two data types can have different meanings. The matrix data type behaves more like matrices in MATLAB.
First install ipython. it is cool to have notebooks!
how to install:
/cube/apps/py3.5.2/bin/pip3 install –upgrade pip
/cube/apps/py3.5.2/bin/pip3 install jupyter
now run it:
/cube/apps/py3.5.2/bin/jupyter notebook
choose new -> python3
and you are good to go…
Ok let’s start with nice tutorial:
Numpy arrays:
For me this was simple and it is very helpful to go through the numpy tutorial …
cool function: arange, linspace, ones, zeros, eye and diag
shaping is also cool, e.g example
import numpy as np
a = np.array([[[1],[2]],[[3],[4]]])
a.shape
(2,2,1) -> what does it mean – It means that we have 2 arrays, with 2 arrays each with one element each…
It took me a while to digest this function … but now I am ok…
to reshape this combination of arrays to more simple thing you can do this:
a.shape = (4,1) # what you did is you asked for 4 arrays one element each instead of 2,2,1, e.g.
array([[1],
[2],
[3],
[4]])
Cool, ha
Everyone is expert in machine learning… Everyone slaps on the resume or CV I know this… On interviews, when you ask candidate some questions about ML you quickly realize they actually do not know anything under the surface… which is painful experience for both parties.
Why is this is beyond me, I guess people think they can make money pretending to be data scientist …
Ok let’s dig into some interesting stuff…
Supervised Learning
When we want to learn from our data by specifying some target variable or value …
Classification – what class an instance of data fall into – simple as that
The target variables could be:
Regression – prediction of a numeric value – do you remember those school days and “best-fit” line ….
Problem facing machine learning algorithms is that there are solutions to problems out there that are not deterministic … example would be motivation in humans… That is hard to model…
I for example used vectored fuzzywuzzy algorithm to mach sentences.
Ok, so what are expert systems! The expert systems are interesting part of machine learning. Basically “expert system” is system that can substitute something that is expert in something.
Think about mathematician or statistician doing something manually on numbers. Well expert system can do the same or better, more precise.
If you measure some subject, you are taking about some rows and columns. Well those columns could be called “features” or “attributes“.
We will have a table “instance” with “features“.
In bellow table we have a patterns of how different races and ethnicity handles (withdraws and deposits annually) for their bank accounts.
| deposit | withdraw | account type | race | ethnicity | |
| 1 | 100000 | 70000 | checking | white | Serbia |
| 2 | 50000 | 45000 | savings | white | America |
| 3 | 20000 | 25000 | checking | black | America |
| 4 | 50000 | 10000 | savings | white | Japan |
| 5 | 200000 | 100000 | checking | brown | Argentina |
The first two features are numeric so they can take a decimal values.
The third feature is binary it can be in this case only 1 or 0.
The fourth column can be enumerated by integers, thus race colors represent numbers, 1,2,3,… n
So we want to do classification on this data set. first we need to come up with classification algorithm and train that algorithm. To do that we need to have a training set, a data .
We have 5 training examples
We have 4 features and one target variable
In classification problem the target variable are called classes and there is assumed to be a finite number of classes.
We will assume that our test set is above.
Machine learning algorithms have a desired level of accuracy. Can we describe that level of accuracy or knowledge representation. It depends. Some algorithms do have knowledge representation some don’t.
Examples of knowledge representation might be:
set of rules
probability distribution
example from the training set
Machine Learning tasks:
So we are working on the classification task.
Classification is prediction of class where the instance of data will be.
Regression is another task in machine learning. Regression is prediction of a numerical value.
Classification and regression are examples of supervised learning …
Opposite of supervised learning we have unsupervised learning. There is no label or target value in data under unsupervised learning . For example clustering, finding statistical values that describe data or reducing the data from many features to a small number of features for visualization purposes are unsupervised learning tasks.
What is important to understand in ML?
How to choose algorithm?
Consider the goal?
What are you trying to get out?
maybe is probability of lowering risk for the bank or similar interests of users who order some product from the retail bank.
So the answer is what data you have or could collect.
Obviously, if you are looking for target values you need to look at supervised learning.
if the value you are looking for is 1/0, yes,no a/b/c/ black/yellow/white then you will use classification. if you are looking into number of values then you are going to use regression, e.a. 0.00 – 100.00, -100 to 100 or +∞ -∞
Opposite is for unsupervised learning …
Trying to fit data into some discrete group would need clustering algorithm. If you want to have some numerical estimate of how strong the fit is in each discrete group, then you should use density estimation algorithm.
It is absolute to know your data. Know your data:
Data features are nominal or continuous?
Are there missing values in the features?
If there are missing values, why is that?
Are there outliers in the data?
Are you looking for something that is very infrequent?
Python I love and it is easy, it really is…
R huh, I am not so sure. I am not super down with syntax and it seems a bit ridged. i do like saving of the session feature… That is a cool feature for any project…
So that being said here is my understanding of R (short)
The R data structures used most frequently are
Each is tailored to a specific data management task, which makes it important to understand how they will interact in your R project.
> testNames testResults testStatus typeof(testStatus)
[1] “logical”
> typeof(testResults)
[1] “double”
> typeof(testNames)
[1] “character”
> testResults[-1]
[1] 102.0 98.2
> testResults[-2]
[1] 100.0 98.2
> testResults[c(TRUE, FALSE, FALSE)]
[1] 100
Ok so to create vectors you can use c (combine function)
Above we created three vectors, we checked types and played with slicing vectors in runtime and playing with inclusions and exclusions of elements (data)
Factors
Something that we can call a features that represent a characteristic with categories of values are known as nominal.
Although it is possible to use a character vector to store nominal data, R provides a data structure specifically for this purpose. A factor is a special case of vector that is solely used to represent categorical or ordinal variables.
Why not use character vectors? An advantage of factors is that the category labels are
stored only once.
For instance, rather than storing SPINDLE, SSD, FLASH, the computer
can store 1, 1, 2, which reduces the size of memory needed to store the same
information. Nominal and numeric data are treated differently. Factors in R coding often is needed to inform an R function to treat categorical data appropriately.
So to compare let’s say some study of storage systems…
nice ordered factor (machine learning wants those)
testBases
>
> testBases
[1] SSD FLASH SPINDLE
Levels: SPINDLE < SSD < FLASH (sequential order preserved from SPINDLE to FLASH)
testBases > “SPINDLE” (some additional tests on validity of grater)
[1] TRUE TRUE FALSE
> testBases > “FLASH”
[1] FALSE FALSE FALSE
> testBases > “SSD”
[1] FALSE TRUE FALSE
List
A list is a data structure, much like a vector, in that it is used for storing an ordered
set of elements. However, where a vector requires all its elements to be the same
type, a list allows different types of elements to be collected.
Lists are often used to store various types of input and output data and sets of
configuration parameters for example machine learning models.
So we can easily poor all these vectors into one list and reference through that object all data values…
tests tests
$name
[1] “test1”$result
[1] 100$status
[1] TRUE$type
[1] SSD
Levels: SPINDLE < SSD < FLASH
For lists in R same applies as vectors, slicing is supported
Notice that in R you can also type list component directly appending $ sign …
tests[2]
$result
[1] 100> tests$type
[1] SSD
Levels: SPINDLE < SSD < FLASH
We can also obtain several parameters from list
> tests[c(“result”, “type”)]
$result
[1] 100$type
[1] SSD
Levels: SPINDLE < SSD < FLASH
Entire datasets could be constructed using lists and lists of lists. For example, you
might consider creating a tests2 and tests3 list, and combining these into
a single list object named testsAll.
Data Frames
The most important R data structure utilized in machine learning is the data
frame, a structure analogous to a spreadsheet or database, since it has both rows
and columns of data. In R terms, a data frame can be understood as a list of:
Each of these having exactly the same number of values. Because the data frame is
literally a list of vector type objects, it combines aspects of both vectors and lists.
catchmonster 