딥러닝에서 손실 함수는 모델의 예측과 실제 값 사이의 차이를 측정하는 중요한 요소. 다양한 종류의 손실 함수가 있으며, 문제의 특성에 따라 적절한 함수를 선택해야 함.
유형 : 회귀
공식 :
$$ \text{MSE} = \frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2 $$
설명:
사용 용도
docker_kubernetes_flask/
├── app/
│ ├── __init__.py
│ ├── app.py
│ └── requirements.txt
├── db/
│ ├── init.sql
│ └── data/ (This will be created by Docker)
├── docker-compose.yml
└── Dockerfile
Docker는 Desktop 설치가 되어 있다고 가정한다.Visual Studio Code를 활용한다.SQL 코드는 데이터베이스와 테이블을 생성하고, 테이블에 데이터를 삽입하는 작업을 수행.
test_db 데이터베이스 생성users 테이블 생성CREATE DATABASE IF NOT EXISTS test_db;
USE test_db;
CREATE TABLE IF NOT EXISTS users (
id INT AUTO_INCREMENT PRIMARY KEY,
name VARCHAR(255) NOT NULL
);
INSERT INTO users (name) VALUES ('Evan');
INSERT INTO users (name) VALUES ('Sara');
INSERT INTO users (name) VALUES ('Lotto');
Python 코드는 Flask 웹 애플리케이션을 설정하여 MySQL 데이터베이스에 연결하고, 사용자 데이터를 JSON 형식으로 반환하는 작업을 수행.from flask import Flask, jsonify
import mysql.connector
import os
app = Flask(__name__)
def get_db_connection():
connection = mysql.connector.connect(
host='mysql',
user='root',
password='example',
database='test_db'
)
return connection
@app.route('/')
def index():
connection = get_db_connection()
cursor = connection.cursor()
cursor.execute('SELECT * FROM users')
users = cursor.fetchall()
cursor.close()
connection.close()
users_list = [{"id": user[0], "name": user[1]} for user in users]
return jsonify(users_list)
if __name__ == '__main__':
app.run(host='0.0.0.0', port=5000)
Flask: Flask 웹 애플리케이션 프레임워크를 불러오기jsonify: 데이터를 JSON 형식으로 변환하여 HTTP 응답으로 반환하는 데 사용mysql.connector: MySQL 데이터베이스에 연결하기 위해 사용from flask import Flask, jsonify
import mysql.connector
import os
__name__은 현재 모듈의 이름을 전달하여 Flask 애플리케이션을 생성하는 데 사용.app = Flask(__name__)
get_db_connection 함수는 MySQL 데이터베이스에 연결하고, 연결 객체를 반환.host, user, password, database 매개변수는 데이터베이스에 연결하기 위한 정보def get_db_connection():
connection = mysql.connector.connect(
host='mysql',
user='root',
password='example',
database='test_db'
)
return connection
@app.route('/'): 해당 Decorator는 URL (’/’)에 대한 요청을 처리하는 index 함수를 정의.get_db_connection을 호출하여 데이터베이스에 연결.SELECT * FROM users 쿼리를 실행하여 users 테이블의 모든 데이터를 가져오기users 데이터를 List Comprehension을 사용하여 딕셔너리 형태로 변환합니다. 각 사용자에 대해 id와 name 키를 가지는 딕셔너리를 생성.
jsonify를 사용하여 JSON 형식으로 반환.@app.route('/')
def index():
connection = get_db_connection()
cursor = connection.cursor()
cursor.execute('SELECT * FROM users')
users = cursor.fetchall()
cursor.close()
connection.close()
users_list = [{"id": user[0], "name": user[1]} for user in users]
return jsonify(users_list)
app.run 메소드를 호출하여 애플리케이션을 시작.host='0.0.0.0': 애플리케이션이 모든 네트워크 인터페이스에서 접근 가능하도록 설정.port=5000: 애플리케이션이 5000번 포트에서 실행되도록 설정.if __name__ == '__main__':
app.run(host='0.0.0.0', port=5000)
Flask
mysql-connector-python
FROM python:3.10-slim
WORKDIR /app
COPY app/requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY app/ .
EXPOSE 5000
CMD ["python", "app.py"]
FROM python:3.10-slim
Docker Official Image 를 선택한다.
!pip install -q streamlit
%%writefile app.py
import streamlit as st
import plotly.graph_objs as go
from plotly.subplots import make_subplots
import seaborn as sns
@st.cache_data
def load_data():
df = sns.load_dataset('tips')
return df
def main():
st.title("Streamlit with Plotly")
tips = load_data()
# 데이터가공
m_tips = tips.loc[tips['sex'] == 'Male', :]
f_tips = tips.loc[tips['sex'] == 'Female', :]
# 시각화 차트
fig = make_subplots(rows = 1,
cols = 2,
subplot_titles=('Male', 'Female'),
shared_yaxes=True,
shared_xaxes=True,
x_title='Total Bill($)'
)
fig.add_trace(go.Scatter(x = m_tips['total_bill'], y = m_tips['tip'], mode='markers'), row=1, col=1)
fig.add_trace(go.Scatter(x = f_tips['total_bill'], y = f_tips['tip'], mode='markers'), row=1, col=2)
fig.update_yaxes(title_text="Tip($)", row=1, col=1)
fig.update_xaxes(range=[0, 60])
fig.update_layout(showlegend=False)
# 중요포인트
# fig.show()
st.plotly_chart(fig, use_container_width=True)
if __name__ == "__main__":
main()
sudo apt update
sudo apt upgrade
sudo apt install gcc make perl
ls
$ {your_location}/google-cloud-sdk/install.sh
Welcome to the Google Cloud CLI!
To help improve the quality of this product, we collect anonymized usage data
and anonymized stacktraces when crashes are encountered; additional information
is available at <https://cloud.google.com/sdk/usage-statistics>. This data is
handled in accordance with our privacy policy
<https://cloud.google.com/terms/cloud-privacy-notice>. You may choose to opt in this
collection now (by choosing 'Y' at the below prompt), or at any time in the
future by running the following command:
gcloud config set disable_usage_reporting false
Do you want to help improve the Google Cloud CLI (y/N)? y
Your current Google Cloud CLI version is: 471.0.0
The latest available version is: 472.0.0
┌────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Components │
├──────────────────┬──────────────────────────────────────────────────────┬──────────────────────────────┬───────────┤
│ Status │ Name │ ID │ Size │
├──────────────────┼──────────────────────────────────────────────────────┼──────────────────────────────┼───────────┤
│ Update Available │ BigQuery Command Line Tool │ bq │ 1.7 MiB │
│ Update Available │ Google Cloud CLI Core Libraries │ core │ 18.4 MiB │
│ Not Installed │ App Engine Go Extensions │ app-engine-go │ 4.5 MiB │
│ Not Installed │ Appctl │ appctl │ 18.5 MiB │
│ Not Installed │ Artifact Registry Go Module Package Helper │ package-go-module │ < 1 MiB │
│ Not Installed │ Cloud Bigtable Command Line Tool │ cbt │ 16.7 MiB │
│ Not Installed │ Cloud Bigtable Emulator │ bigtable │ 7.0 MiB │
│ Not Installed │ Cloud Datastore Emulator │ cloud-datastore-emulator │ 36.2 MiB │
│ Not Installed │ Cloud Firestore Emulator │ cloud-firestore-emulator │ 44.8 MiB │
│ Not Installed │ Cloud Pub/Sub Emulator │ pubsub-emulator │ 63.3 MiB │
│ Not Installed │ Cloud Run Proxy │ cloud-run-proxy │ 11.3 MiB │
│ Not Installed │ Cloud SQL Proxy v2 │ cloud-sql-proxy │ 13.2 MiB │
│ Not Installed │ Google Container Registry's Docker credential helper │ docker-credential-gcr │ │
│ Not Installed │ Kustomize │ kustomize │ 7.4 MiB │
│ Not Installed │ Log Streaming │ log-streaming │ 11.9 MiB │
│ Not Installed │ Minikube │ minikube │ 33.6 MiB │
│ Not Installed │ Nomos CLI │ nomos │ 28.6 MiB │
│ Not Installed │ On-Demand Scanning API extraction helper │ local-extract │ 13.7 MiB │
│ Not Installed │ Skaffold │ skaffold │ 22.8 MiB │
│ Not Installed │ Terraform Tools │ terraform-tools │ 63.6 MiB │
│ Not Installed │ anthos-auth │ anthos-auth │ 20.7 MiB │
│ Not Installed │ config-connector │ config-connector │ 55.6 MiB │
│ Not Installed │ enterprise-certificate-proxy │ enterprise-certificate-proxy │ 8.3 MiB │
│ Not Installed │ gcloud Alpha Commands │ alpha │ < 1 MiB │
│ Not Installed │ gcloud Beta Commands │ beta │ < 1 MiB │
│ Not Installed │ gcloud app Java Extensions │ app-engine-java │ 126.2 MiB │
│ Not Installed │ gcloud app PHP Extensions │ app-engine-php │ 21.9 MiB │
│ Not Installed │ gcloud app Python Extensions │ app-engine-python │ 5.0 MiB │
│ Not Installed │ gcloud app Python Extensions (Extra Libraries) │ app-engine-python-extras │ < 1 MiB │
│ Not Installed │ gke-gcloud-auth-plugin │ gke-gcloud-auth-plugin │ 7.4 MiB │
│ Not Installed │ kpt │ kpt │ 14.4 MiB │
│ Not Installed │ kubectl │ kubectl │ < 1 MiB │
│ Not Installed │ kubectl-oidc │ kubectl-oidc │ 20.7 MiB │
│ Not Installed │ pkg │ pkg │ │
│ Installed │ Cloud Storage Command Line Tool │ gsutil │ 11.3 MiB │
│ Installed │ Google Cloud CRC32C Hash Tool │ gcloud-crc32c │ 1.2 MiB │
└──────────────────┴──────────────────────────────────────────────────────┴──────────────────────────────┴───────────┘
To install or remove components at your current SDK version [471.0.0], run:
$ gcloud components install COMPONENT_ID
$ gcloud components remove COMPONENT_ID
To update your SDK installation to the latest version [472.0.0], run:
$ gcloud components update
Modify profile to update your $PATH and enable shell command completion?
Do you want to continue (Y/n)? y
The Google Cloud SDK installer will now prompt you to update an rc file to bring
the Google Cloud CLIs into your environment.
Enter a path to an rc file to update, or leave blank to use
[/Users/evan/.bash_profile]:
Backing up [/Users/evan/.bash_profile] to [/Users/evan/.bash_profile.backup].
[/Users/evan/.bash_profile] has been updated.
==> Start a new shell for the changes to take effect.
Google Cloud CLI works best with Python 3.11 and certain modules.
Download and run Python 3.11 installer? (Y/n)? Y
Running Python 3.11 installer, you may be prompted for sudo password...
Password: # your_password
installer: Package name is Python
installer: Upgrading at base path /
installer: The upgrade was successful.
Setting up virtual environment
Creating virtualenv...
Installing modules...
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 89.7/89.7 kB 4.6 MB/s eta 0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 59.0/59.0 kB 5.9 MB/s eta 0:00:00
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.0/10.0 MB 33.9 MB/s eta 0:00:00
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 163.8/163.8 kB 16.1 MB/s eta 0:00:00
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.3/5.3 MB 42.0 MB/s eta 0:00:00
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 176.7/176.7 kB 15.4 MB/s eta 0:00:00
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 117.6/117.6 kB 10.4 MB/s eta 0:00:00
Building wheel for crcmod (pyproject.toml) ... done
Updates are available for some Google Cloud CLI components. To install them,
please run:
$ gcloud components update
Virtual env enabled.
For more information on how to get started, please visit:
https://cloud.google.com/sdk/docs/quickstarts
gcloud init 을 실행한다.$ ./google-cloud-sdk/bin/gcloud init
Welcome! This command will take you through the configuration of gcloud.
Your current configuration has been set to: [default]
You can skip diagnostics next time by using the following flag:
gcloud init --skip-diagnostics
Network diagnostic detects and fixes local network connection issues.
Checking network connection...done.
Reachability Check passed.
Network diagnostic passed (1/1 checks passed).
You must log in to continue. Would you like to log in (Y/n)? Y
Your browser has been opened to visit:
https://accounts.google.com/o/oauth2/....S256
You are logged in as: [your_email@gmail.com].
Pick cloud project to use:
[1] chickenbot-a185e
[2] encoded-phalanx-420600
[3] streamlit-gitaction-403105
[4] streamlit-project-416601
[5] Enter a project ID
[6] Create a new project
Please enter numeric choice or text value (must exactly match list item): 2
Welcome to the Google Cloud CLI! Run "gcloud -h" to get the list of available commands.
---
Welcome! This command will take you through the configuration of gcloud.
Settings from your current configuration [default] are:
accessibility:
screen_reader: 'True'
compute:
region: asia-northeast3
zone: asia-northeast3-a
core:
account: your_existing@gmail.com
disable_usage_reporting: 'False'
project: mulcampfp
Pick configuration to use:
[1] Re-initialize this configuration [default] with new settings
[2] Create a new configuration
Please enter your numeric choice: 2
[mulcamp-project-abc] 프로젝트를 입력한다.Enter configuration name. Names start with a lower case letter and contain only lower case letters a-z, digits 0-9, and
hyphens '-': mulcamp-project-abc
Your current configuration has been set to: [mulcamp-project-abc]
You can skip diagnostics next time by using the following flag:
gcloud init --skip-diagnostics
Network diagnostic detects and fixes local network connection issues.
Checking network connection...done.
Reachability Check passed.
Network diagnostic passed (1/1 checks passed).
Choose the account you would like to use to perform operations for this configuration:
[1] your_email@gmail.com
[2] Log in with a new account
Please enter your numeric choice: 1
$ ssh-keygen -t rsa -b 4096 -C "your@gmail.com"
$ cat .ssh/id_rsa.pub
ssh-rsa AAAAB3...Q== your@gmail.com
mkdir -p ~/miniconda3
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda3/miniconda.sh
bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3
rm -rf ~/miniconda3/miniconda.sh
~/miniconda3/bin/conda init bash
~/miniconda3/bin/conda init zsh
pip install django
django-admin --version
5.0.4
django-admin startproject mysite
$ python manage.py makemigrations
$ python manage.py migrate
Operations to perform:
Apply all migrations: admin, auth, contenttypes, sessions
Running migrations:
Applying contenttypes.0001_initial... OK
Applying auth.0001_initial... OK
Applying admin.0001_initial... OK
Applying admin.0002_logentry_remove_auto_add... OK
Applying admin.0003_logentry_add_action_flag_choices... OK
Applying contenttypes.0002_remove_content_type_name... OK
Applying auth.0002_alter_permission_name_max_length... OK
Applying auth.0003_alter_user_email_max_length... OK
Applying auth.0004_alter_user_username_opts... OK
Applying auth.0005_alter_user_last_login_null... OK
Applying auth.0006_require_contenttypes_0002... OK
Applying auth.0007_alter_validators_add_error_messages... OK
Applying auth.0008_alter_user_username_max_length... OK
Applying auth.0009_alter_user_last_name_max_length... OK
Applying auth.0010_alter_group_name_max_length... OK
Applying auth.0011_update_proxy_permissions... OK
Applying auth.0012_alter_user_first_name_max_length... OK
Applying sessions.0001_initial... OK
$ python manage.py runserver
...
April 18, 2024 - 05:13:47
Django version 5.0.4, using settings 'mysite.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
https://dlcdn.apache.org/spark/spark-3.5.1/spark-3.5.1-bin-hadoop3.tgz