스타벅스 아이스 아메리카노를 선물로 보내드리겠습니다.
요한 카를 프리드리히 가우스(1777-1855)가 문제를 냈다고 알려짐
1 + 2 + … + 1000000 까지 해당하는 모든 연속 양수의 합계를 구한다.
두가지 방법이 존재한다.
스타벅스 아이스 아메리카노를 선물로 보내드리겠습니다.PyDataset 라이브러리를 활용합니다.!pip install pydataset
Collecting pydataset
[?25l Downloading https://files.pythonhosted.org/packages/4f/15/548792a1bb9caf6a3affd61c64d306b08c63c8a5a49e2c2d931b67ec2108/pydataset-0.2.0.tar.gz (15.9MB)
[K |████████████████████████████████| 15.9MB 285kB/s
[?25hRequirement already satisfied: pandas in /usr/local/lib/python3.7/dist-packages (from pydataset) (1.1.5)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.7/dist-packages (from pandas->pydataset) (2.8.1)
Requirement already satisfied: numpy>=1.15.4 in /usr/local/lib/python3.7/dist-packages (from pandas->pydataset) (1.19.5)
Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.7/dist-packages (from pandas->pydataset) (2018.9)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.7.3->pandas->pydataset) (1.15.0)
Building wheels for collected packages: pydataset
Building wheel for pydataset (setup.py) ... [?25l[?25hdone
Created wheel for pydataset: filename=pydataset-0.2.0-cp37-none-any.whl size=15939431 sha256=ebe470895a3467fe13c7654021e9108227a6dec8ce6da4f9b4e704520bcd6203
Stored in directory: /root/.cache/pip/wheels/fe/3f/dc/5d02ccc767317191b12d042dd920fcf3432fab74bc7978598b
Successfully built pydataset
Installing collected packages: pydataset
Successfully installed pydataset-0.2.0
from pydataset import data
print(data())
dataset_id title
0 AirPassengers Monthly Airline Passenger Numbers 1949-1960
1 BJsales Sales Data with Leading Indicator
2 BOD Biochemical Oxygen Demand
3 Formaldehyde Determination of Formaldehyde
4 HairEyeColor Hair and Eye Color of Statistics Students
.. ... ...
752 VerbAgg Verbal Aggression item responses
753 cake Breakage Angle of Chocolate Cakes
754 cbpp Contagious bovine pleuropneumonia
755 grouseticks Data on red grouse ticks from Elston et al. 2001
756 sleepstudy Reaction times in a sleep deprivation study
cake = data("cake")
print(cake)
data("cake", show_doc=True)
replicate recipe temperature angle temp
1 1 A 175 42 175
2 1 A 185 46 185
3 1 A 195 47 195
4 1 A 205 39 205
5 1 A 215 53 215
.. ... ... ... ... ...
266 15 C 185 28 185
267 15 C 195 25 195
268 15 C 205 25 205
269 15 C 215 31 215
270 15 C 225 25 225
cake
PyDataset Documentation (adopted from R Documentation. The displayed examples are in R)
## Breakage Angle of Chocolate Cakes
### Description
Data on the breakage angle of chocolate cakes made with three different
recipes and baked at six different temperatures. This is a split-plot design
with the recipes being whole-units and the different temperatures being
applied to sub-units (within replicates). The experimental notes suggest that
the replicate numbering represents temporal ordering.
### Format
A data frame with 270 observations on the following 5 variables.
`replicate`
a factor with levels `1` to `15`
`recipe`
a factor with levels `A`, `B` and `C`
`temperature`
an ordered factor with levels `175` < `185` < `195` < `205` < `215` < `225`
`angle`
a numeric vector giving the angle at which the cake broke.
`temp`
numeric value of the baking temperature (degrees F).
### Details
The `replicate` factor is nested within the `recipe` factor, and `temperature`
is nested within `replicate`.
### Source
Original data were presented in Cook (1938), and reported in Cochran and Cox
(1957, p. 300). Also cited in Lee, Nelder and Pawitan (2006).
### References
Cook, F. E. (1938) _Chocolate cake, I. Optimum baking temperature_. Master's
Thesis, Iowa State College.
Cochran, W. G., and Cox, G. M. (1957) _Experimental designs_, 2nd Ed. New
York, John Wiley \& Sons.
Lee, Y., Nelder, J. A., and Pawitan, Y. (2006) _Generalized linear models with
random effects. Unified analysis via H-likelihood_. Boca Raton, Chapman and
Hall/CRC.
### Examples
str(cake)
## 'temp' is continuous, 'temperature' an ordered factor with 6 levels
(fm1 <- lmer(angle ~ recipe * temperature + (1|recipe:replicate), cake, REML= FALSE))
(fm2 <- lmer(angle ~ recipe + temperature + (1|recipe:replicate), cake, REML= FALSE))
(fm3 <- lmer(angle ~ recipe + temp + (1|recipe:replicate), cake, REML= FALSE))
## and now "choose" :
anova(fm3, fm2, fm1)
스타벅스 아이스 아메리카노를 선물로 보내드리겠습니다.UCI Machine Learning Repository 데이터를 활용해서 MLOps를 구축해본다.$ student_02_2523be913322@cloudshell:~ (qwiklabs-gcp-02-9960bd90e36a)$ gcloud auth list
Credentialed Accounts
ACTIVE ACCOUNT
* student-02-2523be913322@qwiklabs.net
To set the active account, run:
$ gcloud config set account `ACCOUNT`
$ export PROJECT_ID=$(gcloud config get-value core/project)
$ gcloud config set project $PROJECT_ID
$ gcloud services enable \
cloudbuild.googleapis.com \
container.googleapis.com \
cloudresourcemanager.googleapis.com \
iam.googleapis.com \
containerregistry.googleapis.com \
containeranalysis.googleapis.com \
ml.googleapis.com \
dataflow.googleapis.com
$ PROJECT_NUMBER=$(gcloud projects describe $PROJECT_ID --format="value(projectNumber)")
CLOUD_BUILD_SERVICE_ACCOUNT="${PROJECT_NUMBER}@cloudbuild.gserviceaccount.com"
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member serviceAccount:$CLOUD_BUILD_SERVICE_ACCOUNT \
--role roles/editor
Updated IAM policy for project [qwiklabs-gcp-02-9960bd90
e36a].
bindings:
- members:
- serviceAccount:qwiklabs-gcp-02-9960bd90e36a@qwiklabs
-gcp-02-9960bd90e36a.iam.gserviceaccount.com
- user:student-02-2523be913322@qwiklabs.net
role: roles/appengine.appAdmin
- members:
- serviceAccount:qwiklabs-gcp-02-9960bd90e36a@qwiklabs
-gcp-02-9960bd90e36a.iam.gserviceaccount.com
.
.
.
- members:
- serviceAccount:qwiklabs-gcp-02-9960bd90e36a@qwiklabs-gcp-02-9960bd90e36a.iam.gserviceaccount.com
- user:student-02-2523be913322@qwiklabs.net
role: roles/viewer
etag: BwXBj7nBxIk=
version: 1
본격적인 빅쿼리 실습에 앞서서, Python과 연동하는 예제를 준비하였다. 빅쿼리 시작에 앞서서 선행적으로 클라우드 사용을 해야 한다.
위 API를 이용하지 않으면 Python 또는 R과 연동해서 사용할 수는 없다. 자주 쓰는것이 아니라면 비용은 거의 발생하지 않으니 염려하지 않아도 된다. 비용관리에 대한 자세한 내용은 BigQuery 권장사항: 비용 관리에서 확인하기를 바란다.
$ R
bash: R: command not found
이는 환경설정 문제이다. 즉, 이러한 경우에는 여러 솔루션이 있다.
그 중에서 필자는 Fourth Solution: 선택하였다.
$ export PATH="/Library/Frameworks/R.framework/Resources:$PATH"
terminal에서 which R을 실행해본다. 아래와 같이 정상적으로 출력이 된다면, 환경설정은 잘 된 것이다.$ which R
/Library/Frameworks/R.framework/Resources/R
Happy To Code
스타벅스 아이스 아메리카노를 선물로 보내드리겠습니다.astype로 끝낸다.astype에 대한 공식 문서를 살펴본다.temp 데이터를 만든다.import pandas as pd
temp = pd.DataFrame({"A": [0,1,2],
"B": ["0", "1", "2"],
"C": [0.0, 1.0, 2.0]})
temp.info()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 3 entries, 0 to 2
Data columns (total 3 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 A 3 non-null int64
1 B 3 non-null object
2 C 3 non-null float64
dtypes: float64(1), int64(1), object(1)
memory usage: 200.0+ bytes
print(temp)
A B C
0 0 0 0
1 1 1 1
2 2 2 2
dtype: ~ 이다.temp["A"].astype(str)
0 0
1 1
2 2
Name: A, dtype: object
temp["B"].astype(int)
0 0
1 1
2 2
Name: B, dtype: int64
temp["C"].astype(int)
0 0
1 1
2 2
Name: C, dtype: int64
temp["A"] = temp["A"].astype(str)
temp["B"] = temp["B"].astype(int)
temp["C"] = temp["C"].astype(int)
temp.info()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 3 entries, 0 to 2
Data columns (total 3 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 A 3 non-null object
1 B 3 non-null int64
2 C 3 non-null int64
dtypes: int64(2), object(1)
memory usage: 200.0+ bytes
Dtype를 확인할 수 있을 것이다.Happy To Code
공식 문서를 한번 읽어보도록 합니다.ASCII라 합니다..py 파일에서 다음과 같이 시작을 합니다.#!/usr/bin/python
# -*- coding: utf-8 -*-
import os, sys
...
첫줄은 /usr/bin에 있는 파이썬에서 실행한다는 의미.
두번째 줄은 File Encoding 형식을 지정
데이터 분석을 위한 SQL 레시피 교재를 빅쿼리에서 활용해본다.
예제 소스 코드를 열어봅니다.
sql 소스코드로 구성이 되어 있는 것을 확인할 수 있습니다.
저자가 말하는 샘플 데이터 내용은 아래와 같습니다.
이번에는 임의의 SQL 파일을 열어서 확인하도록 합니다.
Table을 생성하는 형태로 구성이 되어 있는 것을 알 수 있습니다.sqlRecipe라고 하였습니다.
!pip install kaggle
Requirement already satisfied: kaggle in /usr/local/lib/python3.7/dist-packages (1.5.12)
Requirement already satisfied: six>=1.10 in /usr/local/lib/python3.7/dist-packages (from kaggle) (1.15.0)
Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from kaggle) (2.23.0)
Requirement already satisfied: urllib3 in /usr/local/lib/python3.7/dist-packages (from kaggle) (1.24.3)
Requirement already satisfied: certifi in /usr/local/lib/python3.7/dist-packages (from kaggle) (2020.12.5)
Requirement already satisfied: python-dateutil in /usr/local/lib/python3.7/dist-packages (from kaggle) (2.8.1)
Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from kaggle) (4.41.1)
Requirement already satisfied: python-slugify in /usr/local/lib/python3.7/dist-packages (from kaggle) (4.0.1)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->kaggle) (2.10)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->kaggle) (3.0.4)
Requirement already satisfied: text-unidecode>=1.3 in /usr/local/lib/python3.7/dist-packages (from python-slugify->kaggle) (1.3)
!mkdir ~/.kaggle
!echo '{"username":"your_id","key":"your_key"}' > ~/.kaggle/kaggle.json
!chmod 600 ~/.kaggle/kaggle.json
!kaggle competitions download -c tabular-playground-series-apr-2021
Warning: Looks like you're using an outdated API Version, please consider updating (server 1.5.12 / client 1.5.4)
Downloading test.csv.zip to /content
0% 0.00/2.07M [00:00<?, ?B/s]
100% 2.07M/2.07M [00:00<00:00, 59.0MB/s]
Downloading train.csv.zip to /content
0% 0.00/2.13M [00:00<?, ?B/s]
100% 2.13M/2.13M [00:00<00:00, 69.3MB/s]
Downloading sample_submission.csv to /content
0% 0.00/879k [00:00<?, ?B/s]
100% 879k/879k [00:00<00:00, 124MB/s]
!ls
sample_data sample_submission.csv test.csv.zip train.csv.zip
!unzip "*.zip"
Archive: train.csv.zip
inflating: train.csv
Archive: test.csv.zip
inflating: test.csv
2 archives were successfully processed.
from google.colab import auth
auth.authenticate_user()
print('Authenticated')
Authenticated
빅쿼리 사용에 앞서서 세팅을 해야 합니다.
Attributes & Methods의 차이점에 대해 이해한다.class Customer:
pass
class <name>: 클래스의 이름을 정의함pass를 입력하면 하나의 empty 클래스를 생성하는 것이다.c1 = Customer()
c2 = Customer()
class Customer:
def identify(self, name):
print("저는 소비자 " + name + " 입니다.")
cust = Customer()
cust.identify("Evan")
저는 소비자 Evan 입니다.
class Customer:
def set_name(self, new_name):
self.name = new_name
.name도 같이 호출 된다.cust2 = Customer() # 이 때에는 .name이 존재하지 않는다.
cust2.set_name("Evan") # 이 때에는 .name이 생성되며, "Evan" 이름이 저장된다.
print(cust2.name) # 정상적으로 호출이 된다
Evan
class Customer:
def set_name(self, new_name):
self.name = new_name
def identify(self):
print("저는 소비자 " + self.name + " 입니다.")
cust = Customer()
cust.set_name("Evan")
cust.identify()
저는 소비자 Evan 입니다.