Tag: Python

How to run HugginFace models in Python

How to run HuggingFace models in Python


Hugging Face

Hugging face is a machine learning community where you can share pre trained models and explore other trained models, which are suited for many use cases.

Many of the models already trained well and ready to use. Without delay get started!

Python setup

Open your Jupiter notebook or for easy startup without python installation on local machine, use Colab.

In this example we are using mistralai/Mistral-7B-v0.1 model. Let’s install the transformers module which is the primary dependency for HuggingFace.

conda install transformers

Following script will work like a charm.

# mistrel model test

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "mistralai/Mistral-7B-v0.1"
tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained(model_id)

text = "Hello my name is"
inputs = tokenizer(text, return_tensors="pt")

outputs = model.generate(**inputs, max_new_tokens=20)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

mistrel model will complete the sentence for you.

Fix Panda dataframe errors for Backtrader

How to fix panda frame for backtrader


Backtrader

BackTrader is a python framework for developing and testing algorithamic trading startegies. It is a great item in the tool kit of stock trader/Alogist.

Fix the Panda DataFrame for BT

I start expirimenting with yfinance data and it is working well with BackTrader as well as Backtesting Framework. I got the Attribute Error : str object has no attribute to_pydatettime while working on historical data from broker API, BREEZE (ICICI DIRECT)

This happends when the date column is not in panda date format. We can replace the colum with panda datatime object it using the following statement.

df['datetime']= pd.to_datetime(df['datetime'])

While I fixed the date error I got another on the datetime column, Attribute Error : int object has no attribute to_pydatettime.

While observing and compare the datafrmae with a yfinance, I found the dataframe I had in a different format, an index column of 0-n, while yfinance dataframe has the date as index column.

Using the Panda builtin functions , able to correct the index column issue and date format.

So I need to replace the numbered index column with datetime column and the problem solved.

Hope this will help someone.

Here is my final code

df =pd.DataFrame(data['Success'])
df =df[['datetime','open','high','low','close','volume']]
df['datetime']= pd.to_datetime(df['datetime']) 
df.set_index('datetime', inplace=True)

How to load an image in Keras from url ?

How to load an image in Keras for machine learning from a url for image classification


How to load an image in Keras for machine learning from a url ?

For loading image in Keras we used to write code like this

img = keras.utils.load_img(path="OIP.jpg",
color_mode = "grayscale",target_size=(28,28,1))

How about using a image URL instead of the above code?

Need to perform couple of things.

  1. Get response from the image url.
  2. Get the image as byte using ByteIO
  3. Covert the image as Grayscale for prediction and apply appropriate resize.
  4. Finally covert the image into a NumPy array.

Let’s wrap all these in a function.

from tensorflow import keras
from io import BytesIO
from PIL import Image
import requests

def loadImage(url):
    response = requests.get(url)
    img_bytes = BytesIO(response.content)
    img = Image.open(img_bytes)
    img = img.convert('L')
 
    img = img.resize((28, 28), Image.NEAREST)
    img = keras.utils.img_to_array(img)

    return img

before using the image for prediction, we need to reshape the image.

url = 'https://edwin-de-jong.github.io/blog/mnist-sequence-data/fig/5.png'
img = loadImage(url)
test_img = img.reshape((1, 784))
img_class = model.predict(test_img)

ML/AI Posts, you may interested.

Get predicted class in Keras Sequential Model

Get predicted class in Keras Sequential Model


 In Keras Sequential Model problems we can get predicted result as output. As a beginner you may note that it is not what you expected.

So how do we get the actual label for that image.?

In image classification we get a predicted result which is pointed to corresponding label. 

In such a case use the NumPy’s argmax function.

img_class = model.predict(test_img)
prediction = img_class[0]
className = np.argmax(prediction, axis=-1)
print("Class : ",className)

Python : list to tuple

Convert Python List to tuple snippet


Convert Python List to tuple snippet

import sqlite3
con = sqlite3.connect('mydb.sqlite')
con.execute("CREATE TABLE IF NOT EXISTS users(id INTEGER ,name TEXT, ecode TEXT);")
con.execute("CREATE TABLE IF NOT EXISTS employees(id INTEGER PRIMARY KEY,ename TEXT, eid INTEGER);")
empnames = [
('jhon', 1),
('martin', 2),
('dev', 3)
]
companies = [
('Foo', 12),
('Bar', 7),
('Moo', 99),
]
comp='SAMSUNG'
comp_id=7006
# argument must be touple
# if one parameter place additional coma
con.execute("INSERT INTO users (id) VALUES (?)", (comp_id,))
con.execute("INSERT INTO users (name,id) VALUES (?,?)", (comp,comp_id))
con.executemany("INSERT or ignore INTO employees (ename,id) VALUES (?,?)", empnames)
con.commit()
# I think this will help you
view raw python_sqlite_and_tuples hosted with ❤ by GitHub

Python SQlite – insertion with tuples and List

Python SQlite – insertion with tuples and List


Python SQlite – insertion with tuples and List

# This code will show how you can convert a list into tuple list
print('Touple meet List')
l1=['Python','C','C++','C#','GO','Dart','Kotlin'] # the list
# List elements to touples
x=[(x,) for x in l1]
print(x)
Out Put
Touple meet List
[('Python',), ('C',), ('C++',), ('C#',), ('GO',), ('Dart',), ('Kotlin',)]
view raw python_list_into_tuple_list hosted with ❤ by GitHub

How to extract keys from json object in Python

How to extract keys from a JavaScript / JSON object in Python


How to extract keys from complex json Object in Python? We can loop through it, but there is a better way.

JSON Object and Keys in Python

Usually we use json module to parse json objects in Python. 

import json
...
data = json.loads(source)

The data variable hold the parsed json object and it can be a list of dictionary values. So how do we extract those keys ? Just like JS, Python also provides the keys method which will extract a list of keys.

countries = list(data.keys())

This will extract all country keys and convert it into a Python list.

Following json post may help you learn more

Install Python pip and setup tools on openSUSE

How to install Python setup tools, pip and python wheel on openSUSE


pip – Pythoners favorite command, is a package management system, which can be used to install and remove python packages from the PYPA(Python Package Authority).

pip and other tools can be installed using the zypper package manager in openSUSE as follow

</p>
sudo zypper install python3-pip python3-setuptools python3-wheel
<p>

For Python version two the following method will do the job

</p>
sudo zypper install python-pip python-setuptools python-wheel
<p>

How to import posts from WordPress to Pelican blog

How to impost WordPress posts to Pelican blog


Pelican is a python powered static site generator which is perfect for creating open blog, project page etc. It is an opensource project, there are many useful plugins and themes to explore.

Compose post with markdown file and host your site with GitHub Page, that make simple.

Import WordPress Posts

You can download posts from WordPress and have it on your GitHub blog.

  • Download WordPress posts using WP Admin tools
  • fetch posts from posts.xml file using command line tool as follows
pelican-import -m markdown --wpfile wp/posts.xml

Here is a demo of GitHub Page made in Pelican

How to pass python list to JavaScript in Flask app

How to pass a python list to JavaScript with ninja template in Flask web app


You can’t use Python list directly into JavaScript, JavaScript lislt/array is not compatible with Python or wise versa. So what to do.

Solution

I have a list of names, which I want pass to a JavaScript in a Flask ninja template. Ninja template allow use to safely convert the python list to Json string then we can pass it to the JavaScript function / just JavaScript

<script type="text/javascript">

        dn = {{ distabr|tojson }}
</script>

You can’t use same route variable in multiple places in same template, it may not serve the purpose

In JavaScript Functions

Suppose you have separate JavaScript function files for some functionality like drawing analytic plots and pie diagram, or some donut chart. So how do we pass value to those functions ?

You can create a JavaScript as above, in your current template, then in function create a script variable to use it as follows.

function init_echarts() {
    var distNames=dn;
}

That’s all you need to know to embed some morris diagram to your flask app.

You should look at Morris JS , it is free and easy to use JavaScript charts library.