Jun 19CachingDistributed Systems combine hundreds of different services. Each of the handles different requests, processing and generating high amounts of data. Some requests could be harder to process and require more resources and time to do that, while they are not so often needed to be recomputed at every page reload…Distributed Systems9 min readDistributed Systems9 min read
Published inCloud Native Daily·Jun 12Service Discovery in Distributed SystemsWhat is Service Discovery? It’s a special service in a distributed system that keeps and makes available information about other services in the system — Working for a couple of years in the data science field and at the same time pursuing a degree in Software Engineering is like dividing yourself into two halves. Half needs to study hard math principles, all types of neural network architectures, data storage, and Business Analysis, while the second…Distributed Systems7 min readDistributed Systems7 min read
Published insoftplus-publication·Aug 13, 2021Bring your data to normal distribution with imperio YeoJohnsonTransformer.One of the biggest problems with data in Data Science is its distribution, it almost every single time isn’t normal. It happens because we cannot have all samples in the world in one data set. However, there exists a bunch of methods that can change that. In one of the…Machie Learning3 min readMachie Learning3 min read
Published insoftplus-publication·Aug 4, 2021Retain Target Information with imperio TargetImputationTransformer.As we said in the article about FrequencyImputationTransoferm, usually, categorical values are replaced by integer numbers. However, this approach is very dangerous for linear models, because of the false correlation that may appear. A step forward from this technique is One Hot Encoding (or Dummy variables). But even Dummy variables have…Machine Learning3 min readMachine Learning3 min read
Published insoftplus-publication·Jul 30, 2021Reduce the outliers influence with imperio SpatialSignTransformer.Outliers are a big problem of almost all types of Machine Learning algorithms. Would be fine if somehow with a data preprocessing technique would be possible to eliminate or at least to reduce their influence on data. …Machine Learning3 min readMachine Learning3 min read
Published insoftplus-publication·Jul 21, 2021Reduce data diversity with imperio LogTransformer.One of the biggest problems with data in Data Science is its distribution, it almost every single time isn’t normal. It happens because we cannot have all samples in the world in one data set. However, there exists a bunch of methods that can change that. …Machine Learning3 min readMachine Learning3 min read
Published insoftplus-publication·Jul 16, 2021Present the categories importance with imperio FrequencyImputationTransformer.Usually, categorical values are replaced by integer numbers. However, this approach is very dangerous for linear models, because of the false correlation that may appear. A step forward from this technique is One Hot Encoding (or Dummy variables). But even Dummy variables have its drawbacks — the matrix is becoming…Artificial Intelligence3 min readArtificial Intelligence3 min read
Published insoftplus-publication·Jul 13, 2021Combine transformers with imperio CombinatorTransformer.Very often we would like to combine different transformations on different columns of our feature matrix in a pipeline. That need motivated us to a special module that gives you this opportunity in imperio — CombinatorTransformer. How CombinatorTransformer works? CombinatorTransformer allows you to apply a specific feature transformation on a specific set of…Machine Learning3 min readMachine Learning3 min read
Published insoftplus-publication·Jun 29, 2021Normalize your data distribution with imperio BoxCoxTransformer.Feature engineering is the process of transforming your input data in such a way that it will be more representative of the Machine Learning Algorithms. However, it is very often forgotten because of the inexistence of an easy-to-use package. That’s why we decided to create the one — imperio, the…Machie Learning3 min readMachie Learning3 min read
Published insoftplus-publication·May 5, 2021Generate synthetic samples using Crucio MTDF.Nowadays the majority of data sets in the industry are unbalanced. Meaning that a class has a higher frequency than others. Very often classifiers in such cases due to the unbalance of the data predict all samples as the most frequent class. To solve this problem we decided at Sigmoid…Unbalanced Data3 min readUnbalanced Data3 min read
