Wednesday, April 18, 2018

Advanced Topics in Python Programming (Not Scripting)

New York Python SQL Bootcamp Coding Classes (Affordable & Cost-effective Machine Learning). Best Free classes in NYC. SQL 101 & Python 101 Classes. Big Data Science Classes for beginners interested in Analytics & Data Science. Weekend part time and full time classes in Manhattan & Queens. 1 on 1 Tutoring also available. Free weekend 2hrs class. Small group courses (2-3 attendees), free takes and 1 on 1 : Python 101, Python Data Science Immersive Python for Data Analytics. VBA Macros Immersive. SQL 1 day Class.Project and Portfolio Oriented on weekends and also free evening classes in NYC. Upload your portfolio to get better job. Best Python Class in NYC. FREE RETAKES.


Object Oriented and Functional Programming Python

  • Different type of Inheritance in Python 
  • Generators, Iterators, Decorators, and Context Managers
  • Working with Threads
  • __new__ vs __init__
  • Dunder
  • Global setting why python is bad for multi threading
  • global optimization for spark
  • Python Generators and Iterator Protocol
  • Python Meta-programming
  • Python object model (at least a passing understanding of metaclasses, slots, and descriptors, as well as how inheritance works), bonus points for recent additions like __prepare__ and __init_subclass__
  • Python's ABCs and inferred types (ie. Iterable, Iterator, Generator, etc
  • Standard library: math, itertools, functools, random, collections, logging, sys, os, and threading/multiprocessing/asyncio


  • Meta Classes
A metaclass is the class of a class. Like a class defines how an instance of the class behaves, a metaclass defines how a class behaves. A class is an instance of a metaclass.
A metaclass is most commonly used as a class-factory. Like you create an instance of the class by calling the class, Python creates a new class (when it executes the 'class' statement) by calling the metaclass

Ref: https://stackoverflow.com/questions/100003/what-are-metaclasses-in-python

  • Abstract Class
Abstract classes: Force a class to implement methods. Abstract classes can contain abstract methods: methods without an implementation. Objects cannot be created from an abstract class. A subclass can implement an abstract class. 



  • Generators, Iterators, Decorators, and Context Managers
A decorator is a function that takes a function as an argument and returns a function as a return value.
an_iterator.__iter__()
itertools is a collection of utilities that make it easy to build an iterator that iterates over sequences in various common ways.
Generators give you the iterator immediately: no access to the underlying data ... if it even exists.
Context Managers: You can encapsulate the setup, error handling and teardown of resources in a few simple steps. The key is to use the with statement.

  • Unit Testing Python
Advanced unit-testing. Mocks, patches, possibly a more advanced library like pytest
Python standard library is called unittest. The principles of unittest are easily portable to other frameworks, like:

  1. unittest
  2. nose or nose2
  3. pytest


pytest supports execution of unittest test cases. The real advantage of pytest comes by writing pytest test cases. pytest test cases are a series of functions in a Python file starting with the name test_.


  • Decorators

Decorators in Python are used to modify or inject code in functions or classes. Using decorators, you can wrap a class or function method call so that a piece of code can be executed before or after the execution of the original code. Decorators can be used to check for permissions, modify or track the arguments passed to a method, logging the calls to a specific method, etc.


  • How is Multithreading achieved in Python? Why is it a bad idea?

Python doesn't allow multi-threading in the truest sense of the word. It has a multi-threading package but if you want to multi-thread to speed your code up, then it's usually not a good idea to use it. Python has a construct called the Global Interpreter Lock (GIL). The GIL makes sure that only one of your 'threads' can execute at any one time. A thread acquires the GIL, does a little work, then passes the GIL onto the next thread. This happens very quickly so to the human eye it may seem like your threads are executing in parallel, but they are really just taking turns using the same CPU core. All this GIL passing adds overhead to execution. This means that if you want to make your code run faster then using the threading package often isn't a good idea.


  • Why a list comprehension is faster than a for loop (which really is to say understand how bytecode is generated, at high level)

https://stackoverflow.com/questions/22108488/are-list-comprehensions-and-functional-functions-faster-than-for-loops

Reference: Advanced Python 3 Programming Techniques By Mark Summerfield

----------------------------------------------------------------------------



Advanced Topics in Python

Topics for Advanced Python usage:
Design Pattern - Using decorators, constructors, classes and data structures in Python
Using Flask framework in the same way as React using the same folder config and other settings. In place of JS we will use Python
Functional Programming in Python and passing on functions in a function. More list comprehensions.


__init__

single underscore vs double underscore


Python Generators and Iterator Protocol
Python Meta-programming
Python Descriptors
Python Decorators (class and method based)
Python Buffering Protocol
Python Comprehensions
Python GIL and multiprocessing and multithreading
Python WSGI protocol
Python Context Managers
Python Design Patterns


Advanced topics in python are:

System Programming (pipes, threads, forks etc.)
Graph Theory (pygraph, Networkx etc)
Polynomial manipulation using python
Linguistics (FSM, Turing manchines etc)
Numerical Computations with Python
Creating Musical Scores With Python
Databases with Python
Python Generators and Iterator Protocol
Python Meta-programming
Python Descriptors
Python Decorators (class and method based)
Python Buffering Protocol
Python Comprehensions
Python GIL and multiprocessing and multi-threading
Python WSGI protocol
Python Context Managers
Python Design Patterns

Third party libraries aside here are some:
metaclasses
writing decorators, generators, iterators
writing context managers
C/c++ extensions
Multiprocessing

  • Understand the python object model (at least a passing understanding of metaclasses, slots, and descriptors, as well as how inheritance works), bonus points for recent additions like __prepare__ and __init_subclass__, but also simpler things like when __new__ is useful
  • Understand python's ABCs and inferred types (ie. Iterable, Iterator, Generator, etc.)
  • Understand the c-level data model (ie. at a high level how an int, a list, and a dict are laid out in memory), bonus points if they are actually correct about the way a dict works in cpython, but simply understanding how an unoptimized dict would work is fine.
  • Know why a list comprehension is faster than a for loop (which really is to say understand how bytecode is generated, at high level)
  • Advanced unittesting. Mocks, patches, possibly a more advanced library like pytest
  • Working knowledge of recent features (async/await, type hints)
  • A decent knowledge of the important parts of the standard library: math, itertools, functools, random, collections, logging, sys, os, and threading/multiprocessing/asyncio (I realize these aren't the same, but still). That is, I'd expect a senior dev to know what contextlib.contextmanager, functools.wraps, and itertools.chain were, and when/why one might want to use them. No need to know every function, but where to look at least.
  • A decent knowledge of some non-standard library modules in the domain. This would highly depend on the field, but scipy stack, django/flask/sqla/jinja2, etc.
  • Know at least one sane way to manage environments. This could be a bare venv, or it could be a docker based solution, or a combination, or pipenv, but something
Coroutines (not just generators)
Decorators
Advanced class construction and topics
C/Cython extensions
Data structures
Ability to debug and profile code
Tests


Table of Contents of the book:  Advanced Python 3 Programming Techniques By Mark Summerfield

Section 1: What This Short Cut Covers 3
Section 2: Branching Using Dictionaries 4
Section 3: Generator Expressions and Functions 5
Section 4: Dynamic Code Execution 9
Section 5: Local and Recursive Functions 16
Section 6: Function and Method Decorators 21
Section 7: Function Annotations 25
Section 8: Controlling Attribute Access 27
Section 9: Functors 31
Section 10: Context Managers 33
Section 11: Descriptors 37
Section 12: Class Decorators 42
Section 13: Abstract Base Classes 45
Section 14: Multiple Inheritance 52
Section 15: Metaclasses 54
Section 16: Functional-Style Programming 59
Section 17: Descriptors with Class Decorators 63
Section 18: About the Author 65

http://buildingskills.itmaybeahack.com/book/python-2.6/html/p03/p03c02_adv_class.html
https://python.swaroopch.com/oop.html
www.shahmoradi.org/ECL2017S/lecture/11-python-advanced-decorator-class
 https://www.reddit.com/r/Python/comments/6wl0qk/what_are_the_top_10_key_featuresadvanced_topics/?st=jg5fbksp&sh=b1e49398
http://python-3-patterns-idioms-test.readthedocs.io/en/latest/Metaprogramming.html
https://jakevdp.github.io/blog/2012/12/01/a-primer-on-python-metaclasses/
http://blog.thedigitalcatonline.com/blog/2014/10/14/decorators-and-metaclasses/

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.