Basic Number - Unit 1 - Counting And Sequences - Steps 2 Learning: Fill & Download for Free

Two main concepts are being currently explored extensively in conjunction with your typical CNN and LSTM (more so with LSTM). These are 1. writable/readable external memory and 2. attention mechanism (soft/hard attention).In a way these two are related, as interacting with external memory requires you to perform some sort of attention to choose where to store memory and which memory to recall to be used.Memory:Interestingly, this path was explored pretty early (2014 is early I guess?…) in Alex Grave’s thesis of Neural Turing Machines, where a LSTM controller learned to write/read with an external memory. It was able to do basic number counting (forward/backwards), copy, etcIt picked up steam recently with with Jason Weston’s End-To-End Memory Networks, which was applied to difficult question answer tasks where short term memory is required, which even a traditional Seq2Seq LSTM performed badly.Here’s a one shot learning paper (One-shot Learning with Memory-Augmented Neural Networks) from Google Deepmind that employs memory network with novel memory unit (least used) doing meta learning. As you can see the external memory unit is the crucial component in this manner.Attention:Attention mechanism has been decently lively in computer vision, and has recently been migrated to NLP applications. See this paper on machine translation: Neural Machine Translation by Jointly Learning to Align and Translate. The basic jist is that instead of just a regular lstm encoder/decoder model, an extra step is taken to allow a soft attention (in the form of softmax) to selectively pick parts of the input sentence to create the output translation. In this sense you skip over irrelevant sequences and focus on key input words that provide the best translation. Note you can run multiple passes of “attention” over the input sequence.When augmented with external memory units, soft attention works well for networks with few memory units, however when this becomes large, soft attention becomes computational heavy as you’re computing some form of a softmax over all memory. Recently there’s been a few papers implementing variant of hard attention, see here: Reinforcement Learning Neural Turing Machines (2015). The catch that hard attention avoids the trouble of looking over all memory units, but it’s non differentiable, thus a reinforcement learning algorithm (REINFORCE policy gradient method) is used to learn to select the optimal memory to recall. You can see the same thing in this very interesting research on dynamically building modular neural networks. (ICLR 2016 paper): Neural Module NetworksFor computer vision application, this is probably the most cited work on attention: Neural Image Caption Generation with Visual Attention (2015) .Here’s a computer vision based hard attention mechanism paper (2015) for online multi object tracking : http://cvgl.stanford.edu/papers/xiang_iccv15.pdfLastly, for the deep reinforcement learning enthusiast, there’s been an interesting implementation of deep memory networks in the RL context in Minecraft! here: Control of Memory, Active Perception, and Action in MinecraftIt seems combining attention and external memory will be the way to go forward as it outperforms traditional benchmarks with vanilla CNN/LSTM implementation across a wide area (NLP, computer vision, maybe RL??)

A computer processor in 10 easy stepsStep 1:Let you imagine a wire with a button in the middle. When you push the button, the electricity goes through, when you release it, the electricity is cut and can't do through anymore. This button is called a transistor and it uses another electricity wire instead of your finger. This is the only tool we have.Step 2:Make your 3 basic circuits.Two buttons on one wire - AND gate - electricity goes through only when both buttons are pushed.A button which is pushed unless you pull it - NOT gates. Electricity stops when you make an action - it reverses your action/signal.Two wires connected on one end so either button is pushed the electricity goes through. - OR gate.Logic AND Gate Tutorial with the Logic AND Gate Truth TableLogic OR Gate Tutorial with Logic OR Gate Truth TableLogic NOT Gate Tutorial with Logic NOT Gate Truth TableStep 3:Now we need to know at least a bit about base-2 math. Look at the table here:Introduction / Binary NumbersWhat we need to know for now is how we can encode number 0-3.We will use two wires. zero means "no current", one means "has current"00 - means zero01 - means one10 - means two11 - means threeStep 4:Let's have 4 wires and let me think of a number 0-3 and the device will select the wire I want by setting up my two wires to the needed number and it allows just the selected signal to go through. It is called a multiplexer.Multiplexer (MUX) and Multiplexing TutorialThere is a device, which works the same way but in the opposite direction it's Demultiplexer (DEMUX) Digital Decoder Tutorial. There is one more called a decoder, but let's keep it simple.So we can now switch signals easily.Step 5:Computers should know how to add numbers. We know how to add with carry from school. The computer should know at least the same, but for the binary numbers.Humans: Addition 4Computers: 0+0 = 0, 1+0=1, 1+1=0 and remember 1 (carry)It's called adder. This is half-adder, but we can connect them together so we can add larger number then 1. It's made from one AND gate and one XOR gate.Subtraction works in a very similar way. Other operations are made from these simple ones. Not that important.Step 6:A computer makes a sequence of commands. We need couple of things for it. First we will learn how to remember a value.It's called a flip-flop and what it does it that you can push zero or one inside (configure the circuit to that state) and keep it as long as you need it. Than you can read it for future use.Step 7:Have you seen the clock signal on the image above? All the technical specifications of mobile phones says the 1.3 gigahertz or similar thing. It refers to this clock signal. It looks a bit like cog from a cogwheel and it works in a similar way. 1 GHz clock generates 1 billion of tics per second and each tick is like a cog which pushes the calculation forward.Step 8:When we put multiple flip-flops next to each other, we can remember larger numbers. This is how memory and registers works. We can retrieve the information from the memory with tools we have. When I want the content of the memory location 5 I will send the signal "5" and with decoder it will be decoded just to "read" signal for the location 5. Just this one will be "opened" and I will receive the content of it to any other location I need. So we reuse wires and the signals open and close the wires so we can change the paths among the parts of the computer.Step 9:Now the computer looks like a pretty dumb thing, how we can control the behavior? We are missing just one device, which is made from the components we know and it is called "finite state machine". You know it for example from the vending machine - look at simple Finite-state machine.It starts with "hello, do you want a drink?", than it switches to "money counting" than "select the drink" and then "make the drink and good bye". Then it starts from the beginning.The device starts in one state. It always reads a number and based on the number it changes its state to another state. The finite state machine we need switches between some simple states "read instruction, decode instruction, calculate, store result".Step 10:A computer. Let's look at one simple implementation.We switch it on. We use one memory location as a program counter. It remembers "where we are". It starts from zero. Finite state machine is in read state, so we open the memory location zero, read a number and decode it as an adding instruction. The state machine therefore opens the arithmetic unit and then switches back to read state, reads the operands and passes them to the arithmetic unit as well. Then it switches to the calculation what basically means that it opens the "gate" from the other side of the arithmetic unit and allows the result to flow back to memory to a location specified in the instruction (let's say it's location 100). Than the state machines switches back to the starting point, it increases the program counter to 1, so we look into memory location 1 for the next instruction. And so on.The computer instructions are very simple like "read a number", "add two numbers", "check if the number is greater then 5", "store number". For programming computers we use so called "high level languages". We can write a command like "flip the image upside-down". This single command is then broken down to often hundreds of the simple computer instructions which are executed one by one very quickly. As mentioned before, nowadays doing about 1 billion instructions per second is nothing special for a mobile phone. But the processor do not contain other parts than the ones described above (beside some technical details). Which is incredible.

The best way to learn Python is to progress through these levels one level at a time. Make sure you completely understand and have extensive hands-on experience at each level before you move to the next chapter. that means you need to actually practice more and more beside watching and reading booksLet’s get start the best way to learn Python language from scratch?Level 0: Basic ProgrammingThis is the level you begin at if you are an absolute python beginner. its mean absolute beginner, I mean someone who has never coded before in Python or any other programming language for that matter.If you know already a language, then you can skip this Level 0.In this level, most of the concepts you will be learning are general programming concepts. The fundamental skills that will boost you as a expert programmer.beside also for the basic programming i recommend Programming Hero apps to learn Python Programming from scratch with every concept example and live codingThis means that these concepts are not really exclusive to Python but can be extended to other programming languages as well.Some of these fundamental concepts areVariables: A variable is any characteristics, number, or quantity that can be measured or counted. A variable may also be called a data itemage, sex, business income and expenses, country of birth, capital expenditure, class grades, eye colour and vehicle type are examples of variables. It is called a variable because the value may vary between data units in a populations and may change in value over time.Examples of VariableThe following are examples of algebraic expressions and equations containing variables.2x + 5 = 10, the variable here is x7y + 10 = 24, the variable here is ya(sq2)+ b(sq2)the variables here are a and bData types: A data type, in programming, is a classification that specifies which type of value a variable has and what type of mathematical, relational or logical operations can be applied to it without causing an error. A string, for example, is a data type that is used to classify text and an integer is a data type used to classify whole numbers.Examples OF Data types:String: (or str or text). Used for a combination of any characters that appear on a keyboard, such as letters, numbers and symbols.Character: (or char). Used for single letters.Integer: (;int). Used for whole numbers.Float: (or Real). Used for numbers that contain decimal points, or for fractions.Boolean: (or bool). Used where data is restricted to True/False or yes/no options.Operations: Operators are symbols that tell the compiler to perform specific mathematical or logical manipulations. In this tutorial , we will try to cover the most commonly used operators in programmingExamples OF Data:1. Arithmetic2. Relational3. Bitwise4. Logical5. Assignment6. Increment7. Miscellaneousfunctions: Every programming language lets you create blocks of code that, when called, perform tasks. Imagine a dog that does the same trick only when asked. Except you do not need dog treats to make your code perform. In programming, these code blocks are called functions.conditionals: Conditionals. Programs generally run from the "top down". A computer will read line one, execute line one, and then go to line two, for example. It is common to include conditional statements to decide if a program should "do something else" if a specific condition is true or falseloops: A for loop is used for iterating over a sequence (that is either a list, a tuple, a dictionary, a set, or a string).This is less like the for keyword in other programming languages, and works more like an iterator method as found in other object-orientated programming languages.With the for loop we can execute a set of statements, once for each item in a list, tuple, set etc.Examples OF loops:Print each fruit in a fruit list:fruits = ["apple", "banana", "cherry"]for x in fruits:print(x)Level 1: Object-oriented ProgrammingIn object-oriented programming, an object refers to a particular instance of a Class. And a Class is like a blueprint of the state and actions that an object can takeStep 1: Learn the concepts of OOPOOP is a programming paradigm, a way of structuring and designing your code.Some examples of these concepts are inheritance, encapsulation, and polymorphism.So make sure you understand these concepts at an abstract level first before you jump into Python’s OOP.Step 2: Learn about Python’s Classes and ObjectsIn this step, you need to apply the abstract concepts you learned in the previous step but specifically in Python.Get comfortable with writing Classes and creating Objects.Step 3: Solve Python problems using OOPThis is a crucial step.In this step, you want to learn how to use OOP to design and structure your code.And as a matter of fact, this step is more of an art than a science. That means the only way to get better is through practice, practice, and more practice.The more you practice, the more you will feel at ease with OOP.Level 2: Concurrent and Parallel ProgrammingNowadays whether you are buying an off-the-shelf laptop or a high-end server for your business, your processor will definitely have multiple cores.And sometimes, your program needs to take advantage of these multiple cores to run things in parallel.This can potentially lead to increased throughput, higher performance, and better responsiveness.But let me be clear about one thing here.If high performance and increased throughput are absolutely crucial, Python wouldn’t be the best language to support parallel programming.In this situation, I would personally prefer golang instead (or good old C).But since this is an article about Python, let’s keep our focus on Python.Before you dive in and write your first parallel program, there are some parallel processing concepts that you should learn about first.Here are some of these concepts.Level 3: Socket Programmingy now you should be very comfortable writing Python code that runs on a single machine.But what if you want to write code that communicates with other machines over a network?If you want to do that, then you need to learn about socket programming.And for that I highly recommend you learn about the basics of computer networks first. Here’s my favorite book.After you learn the basic networking concepts, you can use Python’s libraries to write code on one machine that communicates with code on another.It’s like magic. I still remember the exhilaration I felt the first time I had two laptops communicating back and forth to each other over a Wifi network.Follow these three steps to get started.Step 1: Write an Echo ProgramMake sure they are two different computers and that both of them are connected to your home network.The idea of the Echo program is simple. The client-side reads a message from the user and sends this message to the server over the network.At the server-side, when this message is received, the server echoes the same message back to the client.Think of the Echo program as the Hello World program but for socket programming.After that, you can move on to more complex programs.Step 2: Play around with HTTPOnce you’re comfortable with writing simple TCP client-server applications, you can start using Python’s requests module to send and receive HTTP messages.This is especially useful because the vast majority of web services these days provide an HTTP API interface that you can interact with programmatically. For example, Facebook, Twitter, and Google maps all have HTTP API interfaces that your code can communicate with.And if you feel a little more adventurous and want to take this a bit further, you can also scrape the web with BeautifulSoup.Step 3: Know thy toolsLike any other program, sometimes when you write a networking program, your program will not work from the first attempt.However, debugging networking programs is a little different than debugging regular programs.That’s why you need to equip yourself with the tools necessary to troubleshoot what’s going on.Level 4: Data Structures and Algorithms in PythonIf you reached this level, give yourself a pat on the shoulder.Because by now, you have the skills that enable you to solve a wide variety of problems.However, something is missing.You are still not seasoned enough at writing efficient code.What do I mean by that?For example, you don’t know how to modify your code to make it run faster. You can’t even analyze why it is slow in the first place.This is normal.The knowledge you have learned so far in the previous levels are not enough for you to have a solid understanding of what performance really is, and how to modify your existing code to make it run faster.Don’t believe me? Look at this simple code that calculates the nth Fibonacci number.def fib(n):  if n < 2:  return n  return fib(n-2) + fib(n-1)  print(fib(100)) The code looks simple enough and very straightforward, right?Try using this code to calculate fib(100) [SPOILER ALERT: it will take an extremely long time]Now let’s make a simple modification to the code.def fib(n, d):  if n < 2:  return n  if n not in d:  d[n] = fib(n-2, d) + fib(n-1, d)  return d[n]  print(fib(100, {})) I used what’s called dynamic programming to solve this problem and make it run astronomically faster.Level 5: Advanced PythonIf you want to venture into the territory of Python fluency and take your skills to the next level, then I highly recommend the “Fluent Python” book.This book assumes you already have a solid understanding of the basics of Python.In Fluent Python, some of the concepts that you already learned from introductory books are covered from a different angle, in more detail, and with greater depth.In addition to that, you will learn some new concepts as well.For example, some of the new concepts that you will learn in this book areHigher-order Functions: explains how functions can be used as first-classThissobjects in PythonControl Flow: covers the topic of generators, context managers, coroutines, and concurrencyMetaprogramming: essentially this is writing code that manipulates code. Some of the topics discussed here are decorators and meta-classeshere is all advanced tutorials of python from waterloo universityUniversity of Waterloo OpenCSbest of luck..