Saturday, March 20, 2021
Handy Gal LLC
High quality, professional and efficient.
I do Roofing, Painting Work, Haydman or Any Type of Home Work Such As Windows, Siding, Doors, Drywall? Kimberly will get it done for you.
Kim has an overall 20 years of experience in all phases of exterior and interior home improvements.
She works very hard to offer the same quality work as all the big companies just without the same cost to our customers.
Free Estimates! Let us know how we can help you.
I do have references!
$1M in General Liability Insurance.
Wednesday, June 3, 2020
MOBILE APP DEVELOPMENT NYC
Mobile Development NYC’s mission is to provide a caring and insightful team. We encourage the strong positive mental attitudes of all of our clients and employees to insure your visions, ideas and focus for your mobile development project manifest into fruition. Our services and the commitment of our team provide the support needed to see your vision through till the end.
Our Site - mobileappdevelopmentnyc.site
Contact Us - Info@MobiledevNYC
Shivagan Joshi : +1(929) 356-5046
Shivgan3@Gmail.Com
Our Site - mobileappdevelopmentnyc.site
Contact Us - Info@MobiledevNYC
Shivagan Joshi : +1(929) 356-5046
Shivgan3@Gmail.Com
Thursday, April 2, 2020
Learn Programming NYC's 24*7 Online Help!!
If you require Help in Python/SQL/Hadoop/Data Analytics/VBA, then we are up with 24*7 Online Help on our websites........
For Any queries/assistance please reply/comment on this blogpost.
Monday, March 30, 2020
Monday, March 16, 2020
1-on-1 With Joshi – Customized Online Classes – 20:00 hrs to 22:00 hrs EST
1-on-1 With Joshi –
Customized Online Classes – 20:00 hrs to 22:00 hrs EST (Mon –
Thurs)
No Problem of Place or
time. Choose your course as you want to learn. Learn it online with Joshi at
just $35/hr.
--------------------------1-on-1-with-Joshi------------------------------
All Courses on Offer:
· Bigdata Analytics 101 (Intro to Hadoop & Pypark).
· Blockchain.
· Data Analytics (Excel, VBA, SQL & Python).
· Blockchain & Crypto.
· Exploratory Data Analysis, Charting Pandas MATPLOT.
· MS Excel Features - Data Analytics for Beginners.
· VBA Macros for MS Excel Data Analytics.
· Machine Learning 101 for Non-Programmers.
· Machine Learning on Python Scikit 101 with Ginger Beer (Beginners).
· Python 101 Class For Beginner Non-Programmers.
· Python fundamental Class For Beginner Non-Programmers.
· Python Intermediate Level.
· Pandas 101 Class.
· Python for Finance 101 Class.
· Python For WebScrapping 101 Class.
· Git 101 using Python Class.
· Full Stack Python Django 101 Class.
· Any other course you may like to learn…..
Learn how to analyze,
summarize & visualize data in hands in one on one classes!!
Only in $35 per hour!!!
Why 1-on-1 with Joshi
(online)?
1. Helps to get the much needed personal touch.
2. All doubts are cleared in the class.
3. You can customize the classes according to your
needs.
4. Your personal experience or examples can be shared
and doubts solved.
5. Very economic and efficient classes at your
desktop!
Timings – 20:00 hours to 22:00 hours (EST) between Monday to Thursday.
Our training theme is centered around projects, for
example, your portfolio or even themes you are doing at work. This is very
different from the repetitive courses given by other tutors with a fixed
syllabus. The outcome of such engagement is a product you can use. It
doesn’t matter if you’ve had no programming experience whatsoever. These
courses start programming from scratch, showing you everything from how to set
up as to how to use list comprehensions, and helping you build a solid
foundation for programming. The best part of these courses is that every new
concept is taught with source code slides and practice problems for you to work
through. This learn-by-doing approach is great for beginners who can quickly
learn both Programming and Data Analytics by following these courses.
Instructor:
· Shivgan Joshi (B.Tech, CFA, FRM, PGDF, MBA, MS)
Thursday, August 1, 2019
Python
Python API GET POST DELETE etc
Multi processing in Python - starvation etc
Collections in Python
try except finally
deep and shallow copy
iterators and generators
once iterators it iterated then nothing can be done
erorr in web servers - creating sockets
Multi processing in Python - starvation etc
Collections in Python
try except finally
deep and shallow copy
iterators and generators
once iterators it iterated then nothing can be done
erorr in web servers - creating sockets
Saturday, April 27, 2019
Python Advanced Programmer
https://github.com/rmotr-curriculum/advanced-python-programming/tree/master/unit-1-advanced-oop-inheritance/lesson-3-calculator-with-static-methods/solutions
https://github.com/rmotr-curriculum/advanced-python-programming
https://github.com/PacktPublishing/Expert-Python-Programming_Second-Edition/tree/master/chapter2
https://github.com/PacktPublishing/Expert-Python-Programming_Second-Edition/blob/master/chapter11/cprofile_profiling.py
https://github.com/dongweiming/Expert-Python/blob/master/code/decorator_class.py
https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day%201.md
https://github.com/mbreining/practice-python/blob/master/box_stacking.py
https://github.com/noahgift/functional_intro_to_python/blob/master/Public_SafariOnline_Day2_Part1.ipynb
Saturday, April 20, 2019
Tuesday, April 2, 2019
Spark Machine Leanring Course NYC
Spark PySpark Advanced Python Devops Software Engineering Course By Joshi
Multiple & Logistic Regression in Spark 101
Spark Mlib code
From the code “ from pyspark.ml import regression”
Imported from Pyspark library and not a local code.
Implementing this on your own for practice, created notebook on community databrics:
https://towardsdatascience.com/building-a-linear-regression-with-pyspark-and-mllib-d065c3ba246a?gi=66d5401a1bb7
Regression Spark parameters
Spark Logistic Regression
https://spark.apache.org/docs/1.4.1/api/java/org/apache/spark/ml/regression/LinearRegression.html
Quant Methods in Regression
Maximum likelihood
Log likelihood
Regression Code
https://dziganto.github.io/classes/data%20science/linear%20regression/machine%20learning/object-oriented%20programming/python/Understanding-Object-Oriented-Programming-Through-Machine-Learning/
Git commands 101 Fork, clone, merge branches
Git command primer to merge two branches
Converting Python into Spark code
Running Scipy would be bad idea as it would slow down things as python doesn’t work on JVM. It is suggested to us Scalanlp-Breeze.
https://stackoverflow.com/questions/40370759/using-python-scipy-from-spark-scala
Is Python code faster or should we have small UDF in Scala?
Not an easy and straight forward answer. But Python doesn’t run on JVM so we have to use it as API.
Garbage Collection and Java Serialization decides the speed.
https://changhsinlee.com/pyspark-udf/
Spark Optimization:
https://blog.cloudera.com/blog/2015/04/how-to-translate-from-mapreduce-to-apache-spark-part-2/
https://databricks.gitbooks.io/databricks-spark-knowledge-base/content/best_practices/prefer_reducebykey_over_groupbykey.html
https://stackoverflow.com/questions/32435263/dataframe-join-optimization-broadcast-hash-join/39404486
Partitioning in Spark
https://www.dezyre.com/article/how-data-partitioning-in-spark-helps-achieve-more-parallelism/297
Executioner and Driver - how many gb of memory
How to make partitions in Spark
Make sure the number of partition is at least the number of executors.
https://github.com/vaquarkhan/vaquarkhan/wiki/How-to-calculate-node-and-executors-memory-in-Apache-Spark
https://umbertogriffo.gitbooks.io/apache-spark-best-practices-and-tuning/content/sparksqlshufflepartitions_draft.html
Spark Partitioning data & speed
How to optimally partition data using spark.
PySpark and its Transformation
https://blog.usejournal.com/tutorial-on-pyspark-transformations-and-mlib-7ed289a9e843?gi=192332db80e
MLlib DataFrame-based API
http://spark.apache.org/docs/latest/ml-guide.html
RDD vs DF
https://www.adsquare.com/comparing-performance-of-spark-dataframes-api-to-spark-rdd/
Submitting a job to Pyspark on terminal
# Run a Python application on a Spark standalone cluster
./bin/spark-submit \
--master spark://207.184.161.138:7077 \
examples/src/main/python/pi.py \
1000
https://spark.apache.org/docs/latest/submitting-applications.html
driver-memory 5G
--conf spark.driver.maxResultSize
--conf spark.shuffle.service.enabled
--conf spark.dynamicAllocation.enabled
--conf spark.ui.enabled
--conf spark.speculation
--conf spark.port.maxRetries
--queue root.mde.ste_queue.ste_queue3
--conf spark.kryoserializer.buffer.max
--conf spark.executor.pyspark.memory
--conf spark.executor.memoryOverhead
Submit a job to spark cluster
https://stackoverflow.com/questions/49081211/submit-python-script-into-spark-cluster
Compilation using Maven
Classes & Functions in Python 103
Meta Classes
A metaclass is the class of a class. A class defines how an instance of the class (i.e. an object) behaves while a metaclass defines how a class behaves. A class is an instance of a metaclass.
Skeleton of a class.
https://stackoverflow.com/questions/100003/what-are-metaclasses-in-python
https://realpython.com/python-metaclasses/
Understanding closures
https://www.programiz.com/python-programming/closure
Super
super(GeneralizedLinearRegression, self)
When do we use Super?
Staticmethods
@staticmethod
With staticmethods, neither self (the object instance) nor cls (the class) is implicitly passed as the first argument. They behave like plain functions except that you can call them from an instance or the class.
https://stackoverflow.com/questions/136097/what-is-the-difference-between-staticmethod-and-classmethod
Double Underscore
How to create private classes in Python
Single Underscore
Names, in a class, with a leading underscore are simply to indicate to other programmers that the attribute or method is intended to be private. However, nothing special is done with the name itself.
To quote PEP-8:
_single_leading_underscore: weak "internal use" indicator. E.g. from M import * does not import objects whose name starts with an underscore.
A single leading underscore isn't exactly just a convention: if you use from foobar import *, and module foobar does not define an __all__ list, the names imported from the module do not include those with a leading underscore. Let's say it's mostly a convention, since this case is a pretty obscure corner;-
__foo__: this is just a convention, a way for the Python system to use names that won't conflict with user names.
_foo: this is just a convention, a way for the programmer to indicate that the variable is private (whatever that means in Python).
__foo: this has real meaning: the interpreter replaces this name with _classname__foo as a way to ensure that the name will not overlap with a similar name in another class.
No other form of underscores have meaning in the Python world.
There's no difference between class, variable, global, etc in these conventions.
https://stackoverflow.com/questions/1301346/what-is-the-meaning-of-a-single-and-a-double-underscore-before-an-object-name
Higher order functions & Decorators
Used @total_ordering which is from functools and is a part of higher order functions.
@property
def getConfidenceInterval(self):
Functions being passed and behavior changed.
Decorators with symbol @ are used for decorator to pass in the function in another function and now change its behavior.
https://stackoverflow.com/questions/681953/how-to-decorate-a-class
https://docs.python.org/2/library/functools.html
Unit Testing & Integration Testing in Python 101
Two folders exist.
Python Unit Testing
UNIT TESTING is a level of software testing where individual units/ components of a software are tested. The purpose is to validate that each unit of the software performs as designed. A unit is the smallest testable part of any software. It usually has one or a few inputs and usually a single output.
There are certification exams about automated testing etc.
https://stackoverflow.com/questions/15351546/how-to-import-a-class-from-unittest-in-python
Magic Methods
https://blog.rmotr.com/python-magic-methods-and-getattr-75cf896b3f88?gi=76e08922d80d
Jira / OpenProject (Agile Development) 101
Reporting Agile Project Development & Sprint
It allows you to see all the open questions to know about the meetings and what people are doing
Principles of Agile Development and what are sprints
Jenkins 101
SDLC and Jenkins
Why is DevOPS becoming important?
https://www.tutorialspoint.com/jenkins/jenkins_overview.htm
https://www.dezyre.com/article/how-data-partitioning-in-spark-helps-achieve-more-parallelism/297
Questions
How does the libraries talk to each other?
SAS running in on VM 101 course?
Python API for Spark or Python runs on the container?
How would we create our code to run on cluster?
set PYTHONPATH=%PYTHONPATH%;
C:\Users\xxx\Downloads\python-3.6.0-embed-amd64\
Linux Commands
Source copy
scp -r username@ip/project/ C:/Users/xxx/Documents
Subscribe to:
Posts (Atom)