Tag Archives: R

Hands-On Reinforcement Learning with R


Reinforcement learning (RL) is an integral part of machine learning (ML), and is used to train algorithms. With this book, you’ll learn how to implement reinforcement learning with R, exploring practical examples such as using tabular Q-learning to control robots.

You’ll begin by learning the basic RL concepts, covering the agent-environment interface, Markov Decision Processes (MDPs), and policy gradient methods. You’ll then use R’s libraries to develop a model based on Markov chains. You will also learn how to solve a multi-armed bandit problem using various R packages. By applying dynamic programming and Monte Carlo methods, you will also find the best policy to make predictions. As you progress, you’ll use Temporal Difference (TD) learning for vehicle routing problem applications. Gradually, you’ll apply the concepts you’ve learned to real-world problems, including fraud detection in finance, and TD learning for planning activities in the healthcare sector. You’ll explore deep reinforcement learning using Keras, which uses the power of neural networks to increase RL’s potential. Finally, you’ll discover the scope of RL and explore the challenges in building and deploying machine learning models.

By the end of this book, you’ll be well-versed with RL and have the skills you need to efficiently implement it with R.

  • Understand how to use MDP to manage complex scenarios
  • Solve classic reinforcement learning problems such as the multi-armed bandit model
  • Use dynamic programming for optimal policy searching
  • Adopt Monte Carlo methods for prediction
  • Apply TD learning to search for the best path
  • Use tabular Q-learning to control robots
  • Handle environments using the OpenAI library to simulate real-world applications
  • Develop deep Q-learning algorithms to improve model performance

Hands-On Reinforcement Learning with R

The biggest Christmas sale

My Books on Python, Matlab and R at prices never seen before


A fixture in the Packt calendar, the $5 campaign is something developers across the industry look forward to every year and 2019 isn’t looking to be any different. Packt Publishing always believed in making tech learning as accessible as possible, and by offering every eBook & video for $5 across the site throughout December they give developers the chance to grow their skills, indulge their curiosity and more for less.

You can buy my ebooks and videos for $ 5. These are the titles:

Implement key reinforcement learning algorithms and techniques using different R packages such as the Markov chain, MDP toolbox, contextual, and OpenAI Gym

Discover powerful ways to effectively solve real-world machine learning problems using key libraries including scikit-learn, TensorFlow, and PyTorch

Demonstrate fundamentals of Deep Learning and neural network methodologies using Keras 2.x

A practical guide to mastering reinforcement learning algorithms using Keras

Uncover the power of artificial neural networks by implementing them through R code.

Extract patterns and knowledge from your data in easy way using MATLAB

Build effective regression models in R to extract valuable insights from real data

Click here to get a preview of the available titles

Why is OOP in R so confusing?

Author: Omar Trejo Navarro

Apart from the fact that R is an interpreted language, which can pretty much confuse people used to compiled languages, most programmers would agree that manipulating objects in R can sometimes be a nightmare. This has to do with two significant features of R that make OOP manipulation highly efficient:

  • R has several object models unlike Java, Python and others, which have only one
  • R implements parametric polymorphism, while other OOP (Object-oriented programming) languages generally implement polymorphism from inside an object

Let’s look at these two in a bit more detail.

Various object models

Working with OOP in R is different from what you may see in other languages such as Python, Java, C++, etc. These languages have a single object model. However, R has various object models, meaning that it lets us implement object-oriented systems in various ways. Specifically, R has the following object models:

  • S3
  • S4
  • Reference Classes
  • R6
  • Base Types


In this article, we will look at S3, S4, and R6, since these are the most-used object models in R, but more on that later. Let’s first understand what parametric polymorphism is.

Parametric polymorphism

R implements parametric polymorphism, which implies that methods in R belong to functions, not classes. Parametric polymorphism basically lets you define a generic method or function for types of objects you haven’t yet defined and may never do. This means that you can use the same name for several functions with different sets of arguments and from various classes.

R’s method calls look just like function calls and R must have a mechanism to know which names require simple function calls and which names require method calls. This mechanism is called generics. Generics allow you to register certain names to be treated as methods in R, and they act as dispatchers. When these registered generic functions are called, R looks into a chain of attributes in the object being passed in the call in order to look for functions that match the method call for that object’s type. If it finds a function, it will call it.

You may have noted that the plot() and summary() functions in R may return different results depending on the objects being passed to them (e.g. a data frame or a linear model instance). This is because they are generic functions that implement polymorphism. This way of working provides simple interfaces for users and makes their tasks much simpler. For instance, if you are exploring a new package and you get some kind of result at some point derived from the package, try calling plot(result), and you may be surprised to get some kind of plot that makes sense. This is not common in other OOP languages.

Now that we have a basic idea of what parametric polymorphism is and how it sets R apart from other OOP languages, let’s look at S3, S4, and R6.

R’s most common object models

The S3 object model of R owes its roots to the object model of the S language, the predecessor of R. S’s object model evolved over time, and its third version introduced class attributes, which paved the way for S3. S3 is the fundamental object model of R, and many of R’s own built-in classes are of the S3 type.

Being the least formal object model in R, S3 does lack in some key respects:

  • It has no formal class definitions, thereby leaving no scope for inheritance or encapsulation
  • Polymorphism can only be implemented through generics

Nevertheless, it makes up for what it lacks in by providing quite a lot of flexibility to the programmers. In the words of Hadley Wickham, “S3 has a certain elegance in its minimalism: you can’t take away any part of it and still have a useful object-oriented system“.

However, one of the biggest demerits of S3 is that it doesn’t provide the required safety; it is very easy to create a class in S3, but it may lead to very confusing and hard-to-debug code if not used with utmost care. For example, a programmer could misspell a function and S3 would raise no issues. This is why S4 came into the picture, developed with the goal of adding safety. It keeps the code safe but, on the flip side, introduces a lot of verbosity to the code. It implements most features of modern object-oriented programming languages:

  • Formal class definitions
  • Inheritance
  • Polymorphism (parametric)
  • Encapsulation

In reality, S3 and S4 are really just ways to implement polymorphism for static functions. The R6 package provides a type of class that is similar to R’s Reference Classes, but it is more efficient and doesn’t depend on S4 classes and the methods package, as Reference Classes do.

When Reference Classes were introduced, some users called the new class system R5, following the names of R’s existing class systems S3 and S4. Although Reference Classes are not actually called R5 nowadays, the name of this package and its classes follow the pattern. Despite being first released over 3 years ago, R6 isn’t widely known. However, it is widely used. For example, R6 is used within Shiny and to manage database connections in the dplyr package.

The decision of which object model to use eventually boils down to a trade-off between flexibility, formality and code cleanness.

R Programming by Example

R is a high-level statistical language and is widely used among statisticians and data miners to develop analytical applications. Given the obvious advantages that R brings to the table, it has become imperative for data scientists to learn the essentials of the language.

R Programming by Example is a hands-on guide that helps you develop a strong fundamental base in R by taking you through a series of illustrations and examples. Written by Omar Trejo Navarro, the book starts with the basic concepts and gradually progresses towards more advanced concepts to give you a holistic view of R. Omar is a well-respected data consultant with expertise in applied mathematics and economics.

If you are an aspiring data science professional or statistician and want to learn more about R programming in a practical and engaging manner, R Programming by Example is the book for you!


Regression Analysis with R

Design and develop statistical nodes to identify unique relationships within data at scale

Regression analysis is a statistical process which enables prediction of relationships between variables. The predictions are based on the casual effect of one variable upon another. Regression techniques for modeling and analyzing are employed on large set of data in order to reveal hidden relationship among the variables.

Regression Analysis with R

This book will give you a rundown explaining what regression analysis is, explaining you the process from scratch. The first few chapters give an understanding of what the different types of learning are – supervised and unsupervised, how these learnings differ from each other. We then move to covering the supervised learning in details covering the various aspects of regression analysis. The outline of chapters are arranged in a way that gives a feel of all the steps covered in a data science process – loading the training dataset, handling missing values, EDA on the dataset, transformations and feature engineering, model building, assessing the model fitting and performance, and finally making predictions on unseen datasets. Each chapter starts with explaining the theoretical concepts and once the reader gets comfortable with the theory, we move to the practical examples to support the understanding. The practical examples are illustrated using R code including the different packages in R such as R Stats, Caret and so on. Each chapter is a mix of theory and practical examples.

By the end of this book you will know all the concepts and pain-points related to regression analysis, and you will be able to implement your learning in your projects.

Regression Analysis with R

Indentazione del codice in R


Anche se la struttura del linguaggio R prevede dei particolari delimitatori per alcuni blocchi di programma, risulta comunque utile, l’indentazione del codice in R, per la relativa individuazione.

Ricordiamo a tal proposito che per indentazione del codice s’intende quella tecnica utilizzata nella programmazione attraverso la quale si evidenziano dei blocchi di programma con l’inserimento di una certa quantità di spazio vuoto all’inizio di una riga di testo, allo scopo di aumentarne la leggibilità.

Anche se, come già detto, R prevede opportuni delimitatori per alcune strutture del linguaggio, utilizzeremo l’indentazione stessa per indicare i blocchi nidificati; a tal proposito si possono usare sia una tabulazione, sia un numero arbitrario di spazi bianchi.

Nell’utilizzo di tale tecnica è necessario ricordare delle semplici raccomandazione:

  • il numero di spazi da utilizzare è variabile;
  • tutte le istruzioni del blocco di programma devono presentare lo stesso numero di spazi di indentazione.

In tale ottica utilizzeremo la convenzione che prevede l’esclusivo utilizzo di due spazi per individuare un nuovo blocco e di tralasciare l’uso del tab. 

Per approfondire l’argomento:

Come installare R su Windows


In questo post vedremo come installare R su Windows. Per il sistema operativo Windows, R si presenta come un unico file exe (scaricabile dal sito CRAN), al seguente url:


Tale file può essere facilmente installato con un doppio clic su di esso e seguendo i pochi passi dell’installazione.

Si tratta delle procedure d’installazione automatizzate, i cosiddetti installer, attraverso i quali la fase d’installazione del software si riduce da parte dell’utente alla necessità di dover cliccare, una serie di volte su dei pulsanti con la scritta avanti.


installare R

installare R

Una volta che il processo si è completato, si può iniziare ad utilizzare R tramite l’icona che comparirà sul desktop o tramite il collegamento disponibile nella lista dei programmi utilizzabili sul nostro sistema.

Come creare oggetti in R


In questo post impareremo come creare oggetti in R. Un oggetto può essere creato attraverso l’utilizzo dell’operatore di assegnazione, che è rappresentato da una freccia diretta verso sinistra con un segno meno affianco; questo simbolo può anche essere orientato da sinistra a destra. Vediamo degli esempi per meglio comprenderne il funzionamento:

> variabile<-5
> variabile
[1] 5
> a<-10
> a
[1] 10
> 10->x
> x
[1] 10

Se l’oggetto indicato esiste già, il suo valore è sovrascritto (la modifica interessa solo gli oggetti presenti nella memoria attiva, non i dati presenti sul disco).


Il valore assegnato in questo modo può anche essere il risultato di un’operazione e/o di una funzione:

> x<-5+3
> x
[1] 8
> x<-sqrt(9)
> x
[1] 3

Nel primo caso abbiamo eseguito una semplice somma algebrica, nel secondo abbiamo applicato la funzione radice quadrata.

In realtà come operatore di assegnazione potremmo utilizzare, allo stesso modo l’operatore =, ma si preferisce utilizzare la notazione appena introdotta per rendere il codice più leggibile, eliminando in questo modo ogni possibile sorgente di ambiguità con l’operatore di uguaglianza.

> x=15
> x
[1] 15
> x=10-2
> x
[1] 8
> x=sqrt(49)
> x
[1] 7