After Width: | Height: | Size: 40 KiB |
After Width: | Height: | Size: 99 KiB |
After Width: | Height: | Size: 32 KiB |
After Width: | Height: | Size: 54 KiB |
After Width: | Height: | Size: 29 KiB |
After Width: | Height: | Size: 42 KiB |
After Width: | Height: | Size: 75 KiB |
After Width: | Height: | Size: 18 KiB |
@ -0,0 +1,848 @@ |
|||
## Anxinyun Analyze |
|||
|
|||
安心云数据分析工具, |
|||
|
|||
![image-20210927161719070](imgs/安心云数据分析工具/image-20210927161719070.png) |
|||
|
|||
参考实现优矿: |
|||
|
|||
![image-20210927161703234](imgs/安心云数据分析工具/image-20210927161703234.png) |
|||
|
|||
数据分析工具框图 by Tyr.Liu |
|||
|
|||
![img](imgs/安心云数据分析工具/anxinyun-jupyter-notebook.png) |
|||
|
|||
|
|||
|
|||
启动: |
|||
|
|||
config.js |
|||
|
|||
```js |
|||
{ |
|||
entry: require('./middlewares/orchestrator').entry, |
|||
opts: { |
|||
kubernetes: { |
|||
url: ANXINCLOUD_K8S_API || 'https://10.8.30.157:6443', |
|||
insecureSkipTlsVerify: true, |
|||
version: 'v1', |
|||
promises: true, |
|||
namespace: 'anxinyun', |
|||
auth: { |
|||
bearer: ANXINCLOUD_K8S_AUTH || '......' |
|||
} |
|||
}, |
|||
//runInPod: true, |
|||
apiUrl: ANXINCLOUD_API, |
|||
notebookToken: '6bf509929765366acb8ef066aa30d2cfc57af186a25f229a', |
|||
instanceName: 'anxinyun-jupyter-notebook', |
|||
proxyPort: 18305 |
|||
} |
|||
} |
|||
``` |
|||
|
|||
|
|||
|
|||
```sh |
|||
set NODE_ENV=development&&node server -p 8000 -u http://10.8.30.157:19084 --qnak YwL-KPPPrPFqm5VfCDLSSePi6pa0c0rxbTDGVUSQ --qnsk dFHk_EfTk6ufIaG56h4gzcL3IrAtwl2RkJcl8XuO --qnbkt notebook-test --qndmn http://pcd3v07yz.bkt.clouddn.com |
|||
# -u 数据API地址 |
|||
``` |
|||
|
|||
|
|||
|
|||
## Jupyter Nootbook |
|||
|
|||
![img](imgs/安心云数据分析工具/20180518221933692) |
|||
|
|||
新的名称 Jupyter 由`Julia`、`Python` 和 `R` 组合而成 |
|||
|
|||
安装使用 |
|||
|
|||
```sh |
|||
pip3 install jupyter |
|||
# 使用帮助 |
|||
jupyter notebook -h |
|||
|
|||
# 启动 |
|||
jupyter notebook |
|||
``` |
|||
|
|||
访问 http://localhost:8888/ |
|||
|
|||
![image-20210927091813110](imgs/安心云数据分析工具/image-20210927091813110.png) |
|||
|
|||
|
|||
|
|||
修改文件路径 |
|||
|
|||
```sh |
|||
jupyter notebook --generate-config |
|||
|
|||
# file:///C:/Users/yww08/.jupyter/jupyter_notebook_config.py |
|||
# c.NotebookApp.notebook_dir = 'E:/tmp/notebook' |
|||
``` |
|||
|
|||
|
|||
|
|||
Magic操作(基于IPython) |
|||
|
|||
```python |
|||
%%timeit |
|||
测算整个单元格的运行时间 |
|||
``` |
|||
|
|||
|
|||
|
|||
![image-20210927162140861](imgs/安心云数据分析工具/image-20210927162140861.png) |
|||
|
|||
|
|||
|
|||
|
|||
|
|||
## K8S API 鉴权 |
|||
|
|||
```sh |
|||
kubectl get sa -n anxinyun |
|||
|
|||
# clusterrole.yaml |
|||
kind: ClusterRole |
|||
apiVersion: rbac.authorization.k8s.io/v1 |
|||
metadata: |
|||
namespace: anxinyun |
|||
name: operator |
|||
rules: |
|||
- apiGroups: [""] # "" indicates the core API group |
|||
resources: ["services","pods"] |
|||
verbs: ["get", "watch", "list","create","update","patch"] |
|||
|
|||
|
|||
kubectl create clusterrolebinding operator-pod \ |
|||
--clusterrole=operator \ |
|||
--serviceaccount=anxinyun:default |
|||
|
|||
|
|||
|
|||
fastest@test-master:~$ kubectl get sa -n anxinyun -o yaml |
|||
apiVersion: v1 |
|||
items: |
|||
- apiVersion: v1 |
|||
kind: ServiceAccount |
|||
metadata: |
|||
creationTimestamp: "2020-08-17T10:35:35Z" |
|||
name: default |
|||
namespace: anxinyun |
|||
resourceVersion: "5982" |
|||
selfLink: /api/v1/namespaces/anxinyun/serviceaccounts/default |
|||
uid: a1100eea-19c2-4477-afca-61344353f2e5 |
|||
secrets: |
|||
- name: default-token-zp6cz |
|||
kind: List |
|||
metadata: |
|||
resourceVersion: "" |
|||
selfLink: "" |
|||
fastest@test-master:~$ kubectl describe secret default-token-zp6cz -n anxinyun |
|||
Name: default-token-zp6cz |
|||
Namespace: anxinyun |
|||
Labels: <none> |
|||
Annotations: kubernetes.io/service-account.name: default |
|||
kubernetes.io/service-account.uid: a1100eea-19c2-4477-afca-61344353f2e5 |
|||
|
|||
Type: kubernetes.io/service-account-token |
|||
|
|||
Data |
|||
==== |
|||
ca.crt: 1025 bytes |
|||
namespace: 8 bytes |
|||
token: eyJhbGciOiJSUzI1NiIsImtpZCI6ImFiVlF0Y1NyZjNNTkRVMFVieTNNTzhyVlc5T094Y3J2RmFfYTF6R0pveDQifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJhbnhpbnl1biIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkZWZhdWx0LXRva2VuLXpwNmN6Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImRlZmF1bHQiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJhMTEwMGVlYS0xOWMyLTQ0NzctYWZjYS02MTM0NDM1M2YyZTUiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6YW54aW55dW46ZGVmYXVsdCJ9.elI35PPYtQp-fletleFR7so88Vozk7g8B7oRa1zy2LxSL1m26s8X6SJAipR5uqweNyi8JML3Yo3lPhs6mmzNLxkTRVk1atyXcCSr6J_iPD2dUUaGTL-ZPRYZ1x8Eb2PfugEQM5tf5YXERXqPpEsxTLM83KkI8ogFJQhLG7s-lWZFbvcgmKpCo3lmzuYf-hO0-JngjLhRxptCUqaFx6s8QwQz0dxNn_EvtMXbZm2cTkewJdsFAzczuKtt2sLiJCl5CSRghWAqkP9pBiC2diwDKzz9A0DevG0b3n7J-9_4fPtbXa5zQI60Rg3XVZRof0XNjw5Nze0ee8bn-6XI8yxIug |
|||
fastest@test-master:~$ |
|||
``` |
|||
|
|||
|
|||
|
|||
|
|||
|
|||
## 本地Micro-K8S |
|||
|
|||
安装WSL |
|||
|
|||
适用于 Linux 的 Windows 子系统可让开发人员直接在 Windows 上按原样运行 GNU/Linux 环境(包括大多数命令行工具、实用工具和应用程序),且不会产生传统虚拟机或双启动设置开销。 |
|||
|
|||
[安装 WSL](https://docs.microsoft.com/zh-cn/windows/wsl/setup/environment#set-up-your-linux-user-info) |
|||
|
|||
```sh |
|||
POWERSHELL |
|||
》wsl --install |
|||
|
|||
PS C:\Users\yww08> wsl --list --online |
|||
以下是可安装的有效分发的列表。 |
|||
请使用“wsl --install -d <分发>”安装。 |
|||
|
|||
NAME FRIENDLY NAME |
|||
Ubuntu Ubuntu |
|||
Debian Debian GNU/Linux |
|||
kali-linux Kali Linux Rolling |
|||
openSUSE-42 openSUSE Leap 42 |
|||
SLES-12 SUSE Linux Enterprise Server v12 |
|||
Ubuntu-16.04 Ubuntu 16.04 LTS |
|||
Ubuntu-18.04 Ubuntu 18.04 LTS |
|||
Ubuntu-20.04 Ubuntu 20.04 LTS |
|||
PS C:\Users\yww08> wsl --install -d Ubuntu-18.04 |
|||
正在下载: Ubuntu 18.04 LTS |
|||
[====================== 38.4% ] |
|||
|
|||
创建linux用户 |
|||
Installing, this may take a few minutes... |
|||
Please create a default UNIX user account. The username does not need to match your Windows username. |
|||
For more information visit: https://aka.ms/wslusers |
|||
Enter new UNIX username: yww |
|||
Enter new UNIX password: 123 |
|||
Retype new UNIX password: 123 |
|||
passwd: password updated successfully |
|||
Installation successful! |
|||
To run a command as administrator (user "root"), use "sudo <command>". |
|||
See "man sudo_root" for details. |
|||
|
|||
更新首选包管理器定期更新和升级包 |
|||
sudo apt update && sudo apt upgrade |
|||
|
|||
通过/mnt/c/ 可以访问宿主机上的文件 |
|||
|
|||
``` |
|||
|
|||
设置Windows终端: |
|||
|
|||
![image-20210927163121708](imgs/安心云数据分析工具/image-20210927163121708.png) |
|||
|
|||
`ctrl+shift+d` 新Tab |
|||
|
|||
`alt+shift+d` Split窗口 |
|||
|
|||
`ctrl+shift+w` 关闭 |
|||
|
|||
|
|||
|
|||
```sh |
|||
#安装docker |
|||
apt install docker.io |
|||
|
|||
sudo usermod -aG docker $USER |
|||
|
|||
sudo cgroupfs-mount |
|||
sudo service docker start |
|||
|
|||
#systemctl daemon-reload |
|||
#systemctl restart docker.service |
|||
|
|||
# 上面的安装方法有问题 |
|||
curl https://get.docker.com | sh |
|||
``` |
|||
|
|||
|
|||
|
|||
【MicroK8S】 [Github](https://github.com/ubuntu/microk8s) |
|||
|
|||
Install MicroK8s with: |
|||
|
|||
<font color='red' size='5'>尝试失败了!!!</font> |
|||
|
|||
``` |
|||
snap install microk8s --classic |
|||
``` |
|||
|
|||
MicroK8s includes a `microk8s kubectl` command: |
|||
|
|||
``` |
|||
sudo microk8s kubectl get nodes |
|||
sudo microk8s kubectl get services |
|||
``` |
|||
|
|||
To use MicroK8s with your existing kubectl: |
|||
|
|||
``` |
|||
sudo microk8s kubectl config view --raw > $HOME/.kube/config |
|||
``` |
|||
|
|||
将用户添加如 microk8s用户组,以实现对k8s的访问 |
|||
|
|||
``` |
|||
sudo usermod -a -G microk8s <username> |
|||
``` |
|||
|
|||
Kubernetes插件 |
|||
|
|||
MicroK8s installs a barebones upstream Kubernetes. Additional services like dns and the Kubernetes dashboard can be enabled using the `microk8s enable` command. |
|||
|
|||
``` |
|||
sudo microk8s enable dns dashboard |
|||
``` |
|||
|
|||
Use `microk8s status` to see a list of enabled and available addons. You can find the addon manifests and/or scripts under `${SNAP}/actions/`, with `${SNAP}` pointing by default to `/snap/microk8s/current`. |
|||
|
|||
|
|||
|
|||
**Copy from Kai.Lu** |
|||
|
|||
镜像准备`fetch-images.sh` |
|||
|
|||
```sh |
|||
#!/bin/bash |
|||
images=( |
|||
k8s.gcr.io/pause:3.1=mirrorgooglecontainers/pause-amd64:3.1 |
|||
gcr.io/google_containers/defaultbackend-amd64:1.4=mirrorgooglecontainers/defaultbackend-amd64:1.4 |
|||
k8s.gcr.io/kubernetes-dashboard-amd64:v1.10.1=registry.cn-hangzhou.aliyuncs.com/google_containers/kubernetes-dashboard-amd64:v1.10.1 |
|||
k8s.gcr.io/heapster-influxdb-amd64:v1.3.3=registry.cn-hangzhou.aliyuncs.com/google_containers/heapster-influxdb-amd64:v1.3.3 |
|||
k8s.gcr.io/heapster-amd64:v1.5.2=registry.cn-hangzhou.aliyuncs.com/google_containers/heapster-amd64:v1.5.2 |
|||
k8s.gcr.io/heapster-grafana-amd64:v4.4.3=registry.cn-hangzhou.aliyuncs.com/google_containers/heapster-grafana-amd64:v4.4.3 |
|||
k8s.gcr.io/metrics-server-amd64:v0.3.6=registry.cn-hangzhou.aliyuncs.com/google_containers/metrics-server-amd64:v0.3.6 |
|||
) |
|||
|
|||
OIFS=$IFS; # 保存旧值 |
|||
|
|||
for image in ${images[@]};do |
|||
IFS='=' |
|||
set $image |
|||
docker pull $2 |
|||
docker tag $2 $1 |
|||
docker rmi $2 |
|||
docker save $1 > 1.tar && microk8s.ctr --namespace k8s.io image import 1.tar && rm 1.tar |
|||
IFS=$OIFS; # 还原旧值 |
|||
done |
|||
``` |
|||
|
|||
|
|||
|
|||
```sh |
|||
./fetch-images.sh |
|||
|
|||
microk8s status --wait-ready |
|||
|
|||
alias mk='microk8s.kubectl' |
|||
|
|||
mk get pods -A |
|||
``` |
|||
|
|||
|
|||
|
|||
|
|||
|
|||
## Python |
|||
|
|||
```python |
|||
#!/usr/bin/python |
|||
# -*- coding: UTF-8 -*- |
|||
|
|||
print( "你好,世界" ) |
|||
|
|||
#Python3.X 源码文件默认使用utf-8编码,所以可以正常解析中文,无需指定 UTF-8 编码。 |
|||
|
|||
注释: |
|||
# 单行 |
|||
''' |
|||
多行 |
|||
''' |
|||
|
|||
""" |
|||
多行 |
|||
""" |
|||
|
|||
类型: |
|||
数字 |
|||
字符串 |
|||
列表 list1 = ['Google', 'Runoob', 1997, 2000] |
|||
元祖 (元素值是不允许修改) (tup1 = ('physics', 'chemistry', 1997, 2000)) |
|||
字典 dict1 = { 'abc': 456 } |
|||
集合 a = set('abracadabra') 或者 可以用大括号({})创建集合。注意:如果要创建一个空集合,你必须用 set() 而不是 {} ;后者创建一个空的字典,下一节我们会介绍这个数据结构。 |
|||
|
|||
|
|||
删除元素 del a['k'] |
|||
range |
|||
|
|||
时间: |
|||
import time # 引入time模块 |
|||
|
|||
ticks = time.time() |
|||
print "当前时间戳为:", ticks |
|||
``` |
|||
|
|||
| en((1, 2, 3)) | 3 | 计算元素个数 | |
|||
| ---------------------------- | ---------------------------- | ------------ | |
|||
| (1, 2, 3) + (4, 5, 6) | (1, 2, 3, 4, 5, 6) | 连接 | |
|||
| ('Hi!',) * 4 | ('Hi!', 'Hi!', 'Hi!', 'Hi!') | 复制 | |
|||
| 3 in (1, 2, 3) | True | 元素是否存在 | |
|||
| for x in (1, 2, 3): print x, | 1 2 3 | 迭代 | |
|||
|
|||
笔记 |
|||
|
|||
```python |
|||
a,b=0,1 |
|||
while b<10: |
|||
print(b,end=',') |
|||
a,b=b,a+b |
|||
|
|||
print() |
|||
# while else 循环 |
|||
c=0 |
|||
while c<10: |
|||
print(c) |
|||
c+=2 |
|||
else: |
|||
print("after c=",c) |
|||
|
|||
# set顺序是乱的 |
|||
chars=set('abcdefg') |
|||
for c in chars: |
|||
print(c) |
|||
|
|||
# range([start,]stop[,step]) |
|||
for i in range(5,21,2): |
|||
print(i) |
|||
|
|||
# for array |
|||
a=['a','b','c'] |
|||
for i in range(len(a)): |
|||
print(i,':',a[i]); |
|||
|
|||
# 空语句 可以用pass占位 |
|||
|
|||
## 迭代器 |
|||
a=[1,2,3,4] |
|||
it=iter(a) |
|||
print(next(it)) |
|||
for i in it: |
|||
print('in range:',i) |
|||
|
|||
# while next 写法 |
|||
import sys |
|||
list=[1,2,3,4] |
|||
it = iter(list) # 创建迭代器对象 |
|||
|
|||
while True: |
|||
try: |
|||
print (next(it)) |
|||
except StopIteration: |
|||
print ('finished') |
|||
break |
|||
#sys.exit() |
|||
|
|||
print('is here?') |
|||
|
|||
# 通过yield生成斐波那契数列 |
|||
def fibonacci(n): |
|||
a,b,count=0,1,0 |
|||
while True: |
|||
if (count>n): |
|||
return |
|||
yield a |
|||
a,b=b,a+b |
|||
count+=1 |
|||
|
|||
f=fibonacci(10) |
|||
for fi in f: |
|||
print('f',fi,end=',') |
|||
|
|||
|
|||
# 函数 |
|||
def hello() : |
|||
print("Hello World!") |
|||
|
|||
ret=hello() |
|||
print('ret:'+str(ret)) |
|||
|
|||
''' |
|||
不可变类型:strings,tuples,numbers 作为函数参数类似C++中值传递 |
|||
可变类型: list,dict 作为函数参数类似C++中的引用传递 |
|||
|
|||
参数可以按名称传递、可以有默认值 |
|||
可变长参数如下 |
|||
''' |
|||
def printinfo(arg1,*vartuple): |
|||
print(arg1) |
|||
print(len(vartuple)) |
|||
printinfo(7) |
|||
|
|||
# 加了两个星号 ** 的参数会以字典的形式导入。 |
|||
|
|||
sum=lambda a,b:a+b |
|||
print(sum(1,20)) |
|||
|
|||
# 列表推导式 |
|||
vec=[2,4,6] |
|||
dd=[3*x for x in vec] |
|||
print(dd) |
|||
|
|||
# 字典的便利 |
|||
dics={'name':'ww','age':18} |
|||
for k,v in dics.items(): |
|||
print(k,'=',b) |
|||
|
|||
# 遍历技巧 |
|||
# for i,v in enumerate(list) 同时获得索引和值 |
|||
# zip(list1,list2) 组合两个序列 |
|||
# reversed(seq) 反向 |
|||
# sorted(seq) 排序 |
|||
|
|||
## 类型转换 |
|||
# int(x[, base]) 将x转换为一个整数,base为进制,默认十进制 |
|||
# |
|||
# long(x[, base] ) 将x转换为一个长整数 |
|||
# |
|||
# float(x) 将x转换到一个浮点数 |
|||
# |
|||
# complex(real[, imag]) 创建一个复数 |
|||
# |
|||
# str(x) 将对象 x 转换为字符串 |
|||
# |
|||
# repr(x) 将对象 x 转换为表达式字符串 |
|||
# |
|||
# eval(str) 用来计算在字符串中的有效Python表达式, 并返回一个对象 |
|||
# |
|||
# tuple(s) 将序列 s 转换为一个元组 |
|||
# |
|||
# list(s) 将序列 s 转换为一个列表 |
|||
# |
|||
# set(s) 转换为可变集合 |
|||
# |
|||
# dict(d) 创建一个字典。d 必须是一个序列(key, value) 元组。 |
|||
# |
|||
# frozenset(s) 转换为不可变集合 |
|||
# |
|||
# chr(x) 将一个整数转换为一个字符 |
|||
# |
|||
# unichr(x) 将一个整数转换为Unicode字符 |
|||
# |
|||
# ord(x) 将一个字符转换为它的整数值 |
|||
# |
|||
# hex(x) 将一个整数转换为一个十六进制字符串 |
|||
# |
|||
# oct(x) 将一个整数转换为一个八进制字符串 |
|||
``` |
|||
|
|||
模块学习: |
|||
|
|||
`m.py` |
|||
|
|||
```python |
|||
# !/usr/bin/python3 |
|||
if __name__=='__main__': |
|||
# 程序独立运行 |
|||
pass |
|||
else: |
|||
# 程序被模块调用 |
|||
print('moduled') |
|||
|
|||
def fabonacci(n): |
|||
a,b,c=0,1,0 |
|||
while c<n: |
|||
yield b |
|||
a,b=b,a+b |
|||
c+=1 |
|||
``` |
|||
|
|||
调用模块: |
|||
|
|||
```python |
|||
import m |
|||
|
|||
ret=m.fabonacci(10) |
|||
|
|||
for r in ret: |
|||
print(r) |
|||
``` |
|||
|
|||
|
|||
|
|||
包的概念: |
|||
|
|||
文件夹 包含 `__init__.py` |
|||
|
|||
``` |
|||
sound/ 顶层包 |
|||
__init__.py 初始化 sound 包 |
|||
formats/ 文件格式转换子包 |
|||
__init__.py |
|||
wavread.py |
|||
wavwrite.py |
|||
aiffread.py |
|||
aiffwrite.py |
|||
auread.py |
|||
auwrite.py |
|||
... |
|||
effects/ 声音效果子包 |
|||
__init__.py |
|||
echo.py |
|||
surround.py |
|||
reverse.py |
|||
... |
|||
filters/ filters 子包 |
|||
__init__.py |
|||
equalizer.py |
|||
vocoder.py |
|||
karaoke.py |
|||
... |
|||
``` |
|||
|
|||
导入方法中`from package import item`,item既可以是子模块(子包),也可以是包里面定义的内容(函数或变量) |
|||
|
|||
导入方法中`from sound.effects import *` 如果这个包里面有子模块,需要定义 `__all__`变量来说明 |
|||
|
|||
``` |
|||
__all__ = ['echo','surround','reverse'] |
|||
``` |
|||
|
|||
|
|||
|
|||
输入输出 和 文件操作 |
|||
|
|||
```python |
|||
# rjust 右对齐 |
|||
for x in range(1,11): |
|||
print(repr(x).rjust(2),repr(x*x).rjust(3),repr(x*x*x).rjust(4),end=' ') |
|||
print() |
|||
|
|||
print('name is {0},age {1},alias {alias}'.format('ww',18,alias='peter')) |
|||
|
|||
# 读文件 |
|||
with open('foo.txt','r') as f: |
|||
# 也可以 f.readlines / f.read(length) |
|||
for line in f: |
|||
print (line,end='') |
|||
|
|||
print(f.closed) |
|||
|
|||
# 写文件 |
|||
with open('bar.txt','w') as f: |
|||
for a in range(0,10): |
|||
f.write(str(a)) |
|||
|
|||
f=open('bar.txt','r') |
|||
print(f.readlines()) |
|||
f.close |
|||
|
|||
# 通过pickle实现序列化和反序列化 |
|||
import pickle,pprint |
|||
|
|||
data1={ |
|||
'a':[1,2.0,4+3j], |
|||
'b':('text',u'unicode text'), |
|||
'c':None |
|||
} |
|||
|
|||
selfref_list = [1, 2, 3] |
|||
selfref_list.append(selfref_list) |
|||
|
|||
output=open('data.pk1','wb') |
|||
pickle.dump(data1,output) |
|||
pickle.dump(selfref_list,output,-1) |
|||
output.close() |
|||
|
|||
# 反序列化 |
|||
input=open('data.pk1','rb') |
|||
data1=pickle.load(input) |
|||
pprint.pprint(data1) |
|||
pprint.pprint(pickle.load(input)) |
|||
``` |
|||
|
|||
|
|||
|
|||
错误异常 |
|||
|
|||
<img src='https://static.runoob.com/images/mix/assets-py.webp'> |
|||
|
|||
```python |
|||
try: |
|||
... |
|||
except OSError as err: |
|||
print('OS Error: {0}'.format(err)) |
|||
except (RuntimeError,TypeError,NameError): |
|||
pass |
|||
except: |
|||
raise |
|||
else: # 没有发生异常时执行 |
|||
... |
|||
finally: # 永远执行 |
|||
... |
|||
``` |
|||
|
|||
|
|||
|
|||
面向对象 |
|||
|
|||
```python |
|||
#!/usr/bin/python3 |
|||
|
|||
class people: |
|||
name='' |
|||
|
|||
# 构造函数 |
|||
def __init__(self,n,a): |
|||
self.name=n |
|||
self.age=a |
|||
|
|||
# 类方法 |
|||
def speak(self): |
|||
print("%s speak age %d"%(self.name,self.age)) |
|||
|
|||
# 继承 (同时支持多继承,从左往右的规则搜索父类方法) |
|||
class student(people): |
|||
grade='' |
|||
|
|||
# 私有变量 |
|||
__private_attrs=0 |
|||
|
|||
def __init__(self,n,a,g): |
|||
people.__init__(self,n,a) |
|||
self.grade=g |
|||
|
|||
# 覆盖 |
|||
def speak(self): |
|||
print("{} speak age {} grade {}".format(self.name,self.age,self.grade)) |
|||
|
|||
# 析构函数 |
|||
def __del__(self): |
|||
pass |
|||
|
|||
# 打印 |
|||
def __repr__(self): |
|||
return "myAge:{age}".format(age=self.age) |
|||
|
|||
st = student('ak',16,'g3') |
|||
st.speak() |
|||
super(student,st).speak() # 调用父类已被覆盖的方法 |
|||
print(repr(st)) |
|||
``` |
|||
|
|||
> mirror: http://mirrors.aliyun.com/pypi/simple/ |
|||
|
|||
作用域: |
|||
|
|||
Python 中只有模块(module),类(class)以及函数(def、lambda)才会引入新的作用域,其它的代码块(如 if/elif/else/、try/except、for/while等)是不会引入新的作用域的,也就是说这些语句内定义的变量,外部也可以访问,如下代码: |
|||
|
|||
```python |
|||
|
|||
``` |
|||
|
|||
global 和 nonlocal关键字用于在指定作用域内修改全局或闭包外部作用域内的变量 |
|||
|
|||
|
|||
|
|||
### 标准库 [官方中文文档目录](https://docs.python.org/zh-cn/3.7/library/index.html) |
|||
|
|||
HTTP请求 |
|||
|
|||
```python |
|||
from urllib.request import urlopen |
|||
for line in urlopen('http://baidu.com'): |
|||
line = line.decode('utf-8') |
|||
print(line) |
|||
|
|||
# pip install requests |
|||
import requests |
|||
|
|||
requests=requests.get('http://baidu.com') |
|||
print(requests.content) |
|||
``` |
|||
|
|||
|
|||
|
|||
日志 |
|||
|
|||
```python |
|||
import logging |
|||
|
|||
logging.warning("warnmsg") |
|||
|
|||
log=logging.getLogger('sk') |
|||
|
|||
# logging.debug(msg, *args, **kwargs) |
|||
log.warning('hello') |
|||
|
|||
# 输出 日志级别:日志器名称:日志内容 |
|||
# 默认是指格式 BASIC_FORMAT "%(levelname)s:%(name)s:%(message)s" |
|||
|
|||
LOG_FORMAT = "%(asctime)s - %(levelname)s - %(message)s" |
|||
DATE_FORMAT = "%m/%d/%Y %H:%M:%S %p" |
|||
logging.basicConfig(filename='my.log', level=logging.DEBUG, format=LOG_FORMAT, datefmt=DATE_FORMAT) |
|||
log=logging.getLogger("filelog") |
|||
log.info("this is a log record") |
|||
|
|||
# 打印错误信息和堆栈 |
|||
logging.warning("Some one delete the log file.", exc_info=True, stack_info=True, extra={'user': 'Tom', 'ip':'47.98.53.222'}) |
|||
``` |
|||
|
|||
|
|||
|
|||
多线程 |
|||
|
|||
```python |
|||
from threading import Timer |
|||
import time |
|||
|
|||
# 定时 |
|||
def hello(): |
|||
print('hello') |
|||
|
|||
t=Timer(4.0,hello) |
|||
t.start() |
|||
|
|||
while True: |
|||
time.sleep(1) |
|||
``` |
|||
|
|||
|
|||
|
|||
数学库 |
|||
|
|||
| Quantum Computing | Statistical Computing | Signal Processing | Image Processing | Graphs and Networks | Astronomy Processes | Cognitive Psychology | |
|||
| ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | |
|||
| ![A computer chip.](https://numpy.org/images/content_images/sc_dom_img/quantum_computing.svg) | ![A line graph with the line moving up.](https://numpy.org/images/content_images/sc_dom_img/statistical_computing.svg) | ![A bar chart with positive and negative values.](https://numpy.org/images/content_images/sc_dom_img/signal_processing.svg) | ![An photograph of the mountains.](https://numpy.org/images/content_images/sc_dom_img/image_processing.svg) | ![A simple graph.](https://numpy.org/images/content_images/sc_dom_img/sd6.svg) | ![A telescope.](https://numpy.org/images/content_images/sc_dom_img/astronomy_processes.svg) | ![A human head with gears.](https://numpy.org/images/content_images/sc_dom_img/cognitive_psychology.svg) | |
|||
| [QuTiP](http://qutip.org/) | [Pandas](https://pandas.pydata.org/) | [SciPy](https://www.scipy.org/) | [Scikit-image](https://scikit-image.org/) | [NetworkX](https://networkx.org/) | [AstroPy](https://www.astropy.org/) | [PsychoPy](https://www.psychopy.org/) | |
|||
| [PyQuil](https://pyquil-docs.rigetti.com/en/stable) | [statsmodels](https://github.com/statsmodels/statsmodels) | [PyWavelets](https://pywavelets.readthedocs.io/) | [OpenCV](https://opencv.org/) | [graph-tool](https://graph-tool.skewed.de/) | [SunPy](https://github.com/sunpy/sunpy) | | |
|||
| [Qiskit](https://qiskit.org/) | [Xarray](https://xarray.pydata.org/en/stable/) | [python-control](https://python-control.org/) | [Mahotas](https://mahotas.rtfd.io/) | [igraph](https://igraph.org/python/) | [SpacePy](https://github.com/spacepy/spacepy) | | |
|||
| | [Seaborn](https://github.com/mwaskom/seaborn) | | | [PyGSP](https://pygsp.rtfd.io/) | | | |
|||
| Bioinformatics | Bayesian Inference | Mathematical Analysis | Chemistry | Geoscience | Geographic Processing | Architecture & Engineering | |
|||
| ![A strand of DNA.](https://numpy.org/images/content_images/sc_dom_img/bioinformatics.svg) | ![A graph with a bell-shaped curve.](https://numpy.org/images/content_images/sc_dom_img/bayesian_inference.svg) | ![Four mathematical symbols.](https://numpy.org/images/content_images/sc_dom_img/mathematical_analysis.svg) | ![A test tube.](https://numpy.org/images/content_images/sc_dom_img/chemistry.svg) | ![The Earth.](https://numpy.org/images/content_images/sc_dom_img/geoscience.svg) | ![A map.](https://numpy.org/images/content_images/sc_dom_img/GIS.svg) | ![A microprocessor development board.](https://numpy.org/images/content_images/sc_dom_img/robotics.svg) | |
|||
| [BioPython](https://biopython.org/) | [PyStan](https://pystan.readthedocs.io/en/latest/) | [SciPy](https://www.scipy.org/) | [Cantera](https://cantera.org/) | [Pangeo](https://pangeo.io/) | [Shapely](https://shapely.readthedocs.io/) | [COMPAS](https://compas.dev/) | |
|||
| [Scikit-Bio](http://scikit-bio.org/) | [PyMC3](https://docs.pymc.io/) | [SymPy](https://www.sympy.org/) | [MDAnalysis](https://www.mdanalysis.org/) | [Simpeg](https://simpeg.xyz/) | [GeoPandas](https://geopandas.org/) | [City Energy Analyst](https://cityenergyanalyst.com/) | |
|||
| [PyEnsembl](https://github.com/openvax/pyensembl) | [ArviZ](https://arviz-devs.github.io/arviz/) | [cvxpy](https://github.com/cvxgrp/cvxpy) | [RDKit](https://github.com/rdkit/rdkit) | [ObsPy](https://github.com/obspy/obspy/wiki) | [Folium](https://python-visualization.github.io/folium) | [Sverchok](https://nortikin.github.io/sverchok/) | |
|||
| [ETE](http://etetoolkit.org/) | [emcee](https://emcee.readthedocs.io/) | [FEniCS](https://fenicsproject.org/) | | [Fatiando a Terra](https://www.fatiando.org/) | | | |
|||
|
|||
|
|||
|
|||
- <img src='imgs/安心云数据分析工具/numpylogoicon.png' width=80 align=left> |
|||
|
|||
#### [NumPy](http://numpy.org/) |
|||
|
|||
提供基础的N维数组 |
|||
|
|||
- [![scipy](https://www.scipy.org/_static/images/scipy_med.png)](https://www.scipy.org/scipylib/index.html) |
|||
|
|||
#### [SciPy library](https://www.scipy.org/scipylib/index.html) |
|||
|
|||
科学计算基础包 |
|||
|
|||
- [![matplotlib](https://www.scipy.org/_static/images/matplotlib_med.png)](http://matplotlib.org/) |
|||
|
|||
#### [Matplotlib](http://matplotlib.org/) |
|||
|
|||
综合的2D图形包 |
|||
|
|||
- [![ipython](https://www.scipy.org/_static/images/ipython.png)](http://ipython.org/) |
|||
|
|||
#### [IPython](http://ipython.org/) |
|||
|
|||
增强的交互控制台 |
|||
|
|||
- [![sympy](https://www.scipy.org/_static/images/sympy_logo.png)](http://sympy.org/) |
|||
|
|||
#### [SymPy](http://sympy.org/) |
|||
|
|||
符号数学 |
|||
|
|||
- [![pandas badge](https://www.scipy.org/_static/images/pandas_badge2.jpg)](http://pandas.pydata.org/) |
|||
|
|||
#### [pandas](http://pandas.pydata.org/) |
|||
|
|||
Data structures & analysis 数据结构化和分析工具 |
|||
|
|||
|
|||
|