Category: Python

How to run HugginFace models in Python

How to run HuggingFace models in Python


Hugging Face

Hugging face is a machine learning community where you can share pre trained models and explore other trained models, which are suited for many use cases.

Many of the models already trained well and ready to use. Without delay get started!

Python setup

Open your Jupiter notebook or for easy startup without python installation on local machine, use Colab.

In this example we are using mistralai/Mistral-7B-v0.1 model. Let’s install the transformers module which is the primary dependency for HuggingFace.

conda install transformers

Following script will work like a charm.

# mistrel model test

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "mistralai/Mistral-7B-v0.1"
tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained(model_id)

text = "Hello my name is"
inputs = tokenizer(text, return_tensors="pt")

outputs = model.generate(**inputs, max_new_tokens=20)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

mistrel model will complete the sentence for you.

Fix Panda dataframe errors for Backtrader

How to fix panda frame for backtrader


Backtrader

BackTrader is a python framework for developing and testing algorithamic trading startegies. It is a great item in the tool kit of stock trader/Alogist.

Fix the Panda DataFrame for BT

I start expirimenting with yfinance data and it is working well with BackTrader as well as Backtesting Framework. I got the Attribute Error : str object has no attribute to_pydatettime while working on historical data from broker API, BREEZE (ICICI DIRECT)

This happends when the date column is not in panda date format. We can replace the colum with panda datatime object it using the following statement.

df['datetime']= pd.to_datetime(df['datetime'])

While I fixed the date error I got another on the datetime column, Attribute Error : int object has no attribute to_pydatettime.

While observing and compare the datafrmae with a yfinance, I found the dataframe I had in a different format, an index column of 0-n, while yfinance dataframe has the date as index column.

Using the Panda builtin functions , able to correct the index column issue and date format.

So I need to replace the numbered index column with datetime column and the problem solved.

Hope this will help someone.

Here is my final code

df =pd.DataFrame(data['Success'])
df =df[['datetime','open','high','low','close','volume']]
df['datetime']= pd.to_datetime(df['datetime']) 
df.set_index('datetime', inplace=True)

How to load an image in Keras from url ?

How to load an image in Keras for machine learning from a url for image classification


How to load an image in Keras for machine learning from a url ?

For loading image in Keras we used to write code like this

img = keras.utils.load_img(path="OIP.jpg",
color_mode = "grayscale",target_size=(28,28,1))

How about using a image URL instead of the above code?

Need to perform couple of things.

  1. Get response from the image url.
  2. Get the image as byte using ByteIO
  3. Covert the image as Grayscale for prediction and apply appropriate resize.
  4. Finally covert the image into a NumPy array.

Let’s wrap all these in a function.

from tensorflow import keras
from io import BytesIO
from PIL import Image
import requests

def loadImage(url):
    response = requests.get(url)
    img_bytes = BytesIO(response.content)
    img = Image.open(img_bytes)
    img = img.convert('L')
 
    img = img.resize((28, 28), Image.NEAREST)
    img = keras.utils.img_to_array(img)

    return img

before using the image for prediction, we need to reshape the image.

url = 'https://edwin-de-jong.github.io/blog/mnist-sequence-data/fig/5.png'
img = loadImage(url)
test_img = img.reshape((1, 784))
img_class = model.predict(test_img)

ML/AI Posts, you may interested.

Get predicted class in Keras Sequential Model

Get predicted class in Keras Sequential Model


 In Keras Sequential Model problems we can get predicted result as output. As a beginner you may note that it is not what you expected.

So how do we get the actual label for that image.?

In image classification we get a predicted result which is pointed to corresponding label. 

In such a case use the NumPy’s argmax function.

img_class = model.predict(test_img)
prediction = img_class[0]
className = np.argmax(prediction, axis=-1)
print("Class : ",className)

Python : list to tuple

Convert Python List to tuple snippet


Convert Python List to tuple snippet

import sqlite3
con = sqlite3.connect('mydb.sqlite')
con.execute("CREATE TABLE IF NOT EXISTS users(id INTEGER ,name TEXT, ecode TEXT);")
con.execute("CREATE TABLE IF NOT EXISTS employees(id INTEGER PRIMARY KEY,ename TEXT, eid INTEGER);")
empnames = [
('jhon', 1),
('martin', 2),
('dev', 3)
]
companies = [
('Foo', 12),
('Bar', 7),
('Moo', 99),
]
comp='SAMSUNG'
comp_id=7006
# argument must be touple
# if one parameter place additional coma
con.execute("INSERT INTO users (id) VALUES (?)", (comp_id,))
con.execute("INSERT INTO users (name,id) VALUES (?,?)", (comp,comp_id))
con.executemany("INSERT or ignore INTO employees (ename,id) VALUES (?,?)", empnames)
con.commit()
# I think this will help you
view raw python_sqlite_and_tuples hosted with ❤ by GitHub

Python SQlite – insertion with tuples and List

Python SQlite – insertion with tuples and List


Python SQlite – insertion with tuples and List

# This code will show how you can convert a list into tuple list
print('Touple meet List')
l1=['Python','C','C++','C#','GO','Dart','Kotlin'] # the list
# List elements to touples
x=[(x,) for x in l1]
print(x)
Out Put
Touple meet List
[('Python',), ('C',), ('C++',), ('C#',), ('GO',), ('Dart',), ('Kotlin',)]
view raw python_list_into_tuple_list hosted with ❤ by GitHub

How to extract keys from json object in Python

How to extract keys from a JavaScript / JSON object in Python


How to extract keys from complex json Object in Python? We can loop through it, but there is a better way.

JSON Object and Keys in Python

Usually we use json module to parse json objects in Python. 

import json
...
data = json.loads(source)

The data variable hold the parsed json object and it can be a list of dictionary values. So how do we extract those keys ? Just like JS, Python also provides the keys method which will extract a list of keys.

countries = list(data.keys())

This will extract all country keys and convert it into a Python list.

Following json post may help you learn more

How to fetch API data in Python

How to make an API call and fetch json data in Python


API allows programmers to share the data in an organization or network. Modern apps may use multiple API’s too. So do we access API data in Python apps ?

Requirements

  • URLLIB3
  • JSON

In Most case data is shared as JSON , so we need to use json package to read the data, and urllib3 library for access and make request .

Function for fetch data

It is wise to create function for fetching data from API , so that you can reuse the function to read data from another API.

Here is the Python function to read data from an API end point .

</p>
from pip._vendor import urllib3
import json

def apicall(url):
    try:
        http = urllib3.PoolManager()
        weburl = http.request('GET', url)
        data = weburl.data.decode('utf-8')
        return data
    except urllib3.exceptions:
        print('Error')
    return 
<p>

Using the function

</p>
        urldata = 'https://api.mysite.com/somjsom.json'
        data = apicall(urldata)
        j_datax = json.loads(data) 
<p>

The apicall function will return the data in json format

Following Python post deserve a good read

Install Python pip and setup tools on openSUSE

How to install Python setup tools, pip and python wheel on openSUSE


pip – Pythoners favorite command, is a package management system, which can be used to install and remove python packages from the PYPA(Python Package Authority).

pip and other tools can be installed using the zypper package manager in openSUSE as follow

</p>
sudo zypper install python3-pip python3-setuptools python3-wheel
<p>

For Python version two the following method will do the job

</p>
sudo zypper install python-pip python-setuptools python-wheel
<p>

Using pelican themes


Pelican is a Python based static site/webpage generator, which is famous among opensource and python lovers.

There are plenty of beautiful themes to style pelican blog site contributed by the awesome community, you can have a look at them @ pelican-themes.

Editor’s Picked themes

I have a hand picked themes for bloggers and for documenting projects

  1. Elegant
  2. Flex
  3. Iris
  4. lazystrap
  5. Pelican-clean-blog
  6. Peli-Kiera
  7. Photowall
  8. Tuxlite-tbs

When it come for blog I have Flex and Elegant in my mind. Elegant is my favorite, which is come with many features and good typography and it is good for blog, portfolio page etc.

How to get themes

You can clone the theme repository to you local machine or you can clone individual theme of your choice as follows

</p>
git clone https://github.com/alexandrevicenzi/Flex.git
<p>

How to use themes in your project

To use themes in your project, create a folder named themes , inside the site folder where you can see content folder and place your theme folder inside it.

Now go to your pelicanconfig.py / settings.py file and add the following line

</p>
THEME = 'themes/elegant'
<p>

Save changes and hit pelican content and pelican –listen