SQL> CREATE TABLESPACE myts DATAFILE 'C:\oracle\oradata\MYORACLE\myts.dbf' SIZE 100M AUTOEXTEND ON NEXT 5M;
테이블스페이스가 생성되었습니다.
SQL> CREATE USER ora_user IDENTIFIED BY evan DEFAULT TABLESPACE MYTS TEMPORARY TABLESPACE TEMP;
사용자가 생성되었습니다.
SQL> GRANT DBA TO ora_user;
권한이 부여되었습니다.
SQL> connect ora_user/evan
연결되었습니다.
SQL> select user from dual;
USER
--------------------------------------------------------------------------------
ORA_USER
Spark Submit을 활용한다.$ pwd
/mnt/c/hadoop/pyskt_tutorial
evan@evan:/mnt/c/hadoop/pyskt_tutorial$ virtualenv venv
evan@evan:/mnt/c/hadoop/pyskt_tutorial$ source venv/bin/activate
(venv) evan@evan:/mnt/c/hadoop/pyskt_tutorial$
(venv) evan@evan:/mnt/c/hadoop/pyskt_tutorial$ pip install pyspark
Requirement already satisfied: pyspark in ./venv/lib/python3.8/site-packages (3.2.1)
Requirement already satisfied: py4j==0.10.9.3 in ./venv/lib/python3.8/site-packages (from pyspark) (0.10.9.3)
(venv) evan@evan:/mnt/c/hadoop/pyskt_tutorial$ mkdir data && cd data
(venv) evan@evan:/mnt/c/hadoop/pyskt_tutorial/data$ vi README.md
This program just counts the number of lines containing ‘a’ and the number containing ‘b’ in a text file. Note that you’ll need to replace YOUR_SPARK_HOME with the location where Spark is installed. As with the Scala and Java examples, we use a SparkSession to create Datasets. For applications that use custom classes or third-party libraries, we can also add code dependencies to spark-submit through its --py-files argument by packaging them into a .zip file (see spark-submit --help for details). SimpleApp is simple enough that we do not need to specify any code dependencies.
We can run this application using the bin/spark-submit script:
from pyspark.sql import SparkSession
logFile = "data/README.md" # Should be some file on your system
spark = SparkSession.builder.appName("SimpleApp").getOrCreate()
logData = spark.read.text(logFile).cache()
numAs = logData.filter(logData.value.contains('a')).count()
numBs = logData.filter(logData.value.contains('b')).count()
print("Lines with a: %i, lines with b: %i" % (numAs, numBs))
input("Typing....")
spark.stop()
bin/spark-submitscript라 부르기도 한다.
(venv) evan@evan:/mnt/c/hadoop/pyskt_tutorial$ $SPARK_HOME/bin/spark-submit --master local[4] /mnt/c/hadoop/pyskt_tutorial/SimpleApp.py
$ sudo apt-get install openjdk-8-jdk
$ sudo wget https://archive.apache.org/dist/spark/spark-3.2.0/spark-3.2.0-bin-hadoop3.2.tgz
$ sudo tar -xvzf spark-3.2.0-bin-hadoop3.2.tgz
evan@evan:/mnt/c/hadoop$ pwd
/mnt/c/hadoop
evan@evan:/mnt/c/hadoop$ ls
spark-3.2.0-bin-hadoop3.2 spark-3.2.0-bin-hadoop3.2.tgz
vi ~/.bashrc 파일을 열고 다음과 같이 코드를 작성한다.
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
export SPARK_HOME=/mnt/c/hadoop/spark-3.2.0-bin-hadoop3.2
export PATH=$JAVA_HOME/bin:$PATH
export PATH=$SPARK_HOME/sbin:$PATH
export PYSPARK_PYTHON=/usr/bin/python3
evan@evan:/mnt/c/hadoop/spark-3.2.0-bin-hadoop3.2$ pwd
/mnt/c/hadoop/spark-3.2.0-bin-hadoop3.2
evan@evan:/mnt/c/hadoop/spark-3.2.0-bin-hadoop3.2$ pysaprk
Python 3.8.10 (default, Mar 15 2022, 12:22:08)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
22/04/19 19:39:57 WARN Utils: Your hostname, evan resolves to a loopback address: 127.0.1.1; using 172.17.233.202 instead (on interface eth0)
22/04/19 19:39:57 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
22/04/19 19:39:59 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/__ / .__/\_,_/_/ /_/\_\ version 3.2.0
/_/
Using Python version 3.8.10 (default, Mar 15 2022 12:22:08)
Spark context Web UI available at http://172.17.233.202:4040
Spark context available as 'sc' (master = local[*], app id = local-1650364801421).
SparkSession available as 'spark'.
>>> rd = sc.textFile("README.md")
>>> rd.count()
109
Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('[https://community.chocolatey.org/install.ps1](https://community.chocolatey.org/install.ps1)'))Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.
새로운 크로스 플랫폼 PowerShell 사용 https://aka.ms/pscore6
PS C:\WINDOWS\system32> Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://community.chocolatey.org/install.ps1'))
Forcing web requests to allow TLS v1.2 (Required for requests to Chocolatey.org)
Getting latest version of the Chocolatey package for download.
Not using proxy.
Getting Chocolatey from https://community.chocolatey.org/api/v2/package/chocolatey/1.1.0.
Downloading https://community.chocolatey.org/api/v2/package/chocolatey/1.1.0 to C:\Users\512-24\AppData\Local\Temp\chocolatey\chocoInstall\chocolatey.zip
Not using proxy.
Extracting C:\Users\512-24\AppData\Local\Temp\chocolatey\chocoInstall\chocolatey.zip to C:\Users\512-24\AppData\Local\Temp\chocolatey\chocoInstall
Installing Chocolatey on the local machine
Creating ChocolateyInstall as an environment variable (targeting 'Machine')
Setting ChocolateyInstall to 'C:\ProgramData\chocolatey'
WARNING: It's very likely you will need to close and reopen your shell
before you can use choco.
Restricting write permissions to Administrators
We are setting up the Chocolatey package repository.
The packages themselves go to 'C:\ProgramData\chocolatey\lib'
(i.e. C:\ProgramData\chocolatey\lib\yourPackageName).
A shim file for the command line goes to 'C:\ProgramData\chocolatey\bin'
and points to an executable in 'C:\ProgramData\chocolatey\lib\yourPackageName'.
Creating Chocolatey folders if they do not already exist.
WARNING: You can safely ignore errors related to missing log files when
upgrading from a version of Chocolatey less than 0.9.9.
'Batch file could not be found' is also safe to ignore.
'The system cannot find the file specified' - also safe.
chocolatey.nupkg file not installed in lib.
Attempting to locate it from bootstrapper.
PATH environment variable does not have C:\ProgramData\chocolatey\bin in it. Adding...
경고: Not setting tab completion: Profile file does not exist at
'C:\Users\512-24\Documents\WindowsPowerShell\Microsoft.PowerShell_profile.ps1'.
Chocolatey (choco.exe) is now ready.
You can call choco from anywhere, command line or powershell by typing choco.
Run choco /? for a list of functions.
You may need to shut down and restart powershell and/or consoles
first prior to using choco.
Ensuring Chocolatey commands are on the path
Ensuring chocolatey.nupkg is in the lib folder
PS C:\WINDOWS\system32>
$ choco install hugo-extended -confirm
$ hugo version
hugo v0.97.0-c07f3626e7c8160943591f4d209977efa02c3dca windows/amd64 BuildDate=2022-04-14T08:45:07Z VendorInfo=gohugoio
PS C:\WINDOWS\system32>
hugo new site your_blog_namemyblog로 지정했다.PS C:\Users\your_name\Desktop> hugo new site myblog
Congratulations! Your new Hugo site is created in C:\Users\512-24\Desktop\myblog.
Just a few more steps and you're ready to go:
1. Download a theme into the same-named folder.
Choose a theme from https://themes.gohugo.io/ or
create your own with the "hugo new theme <THEMENAME>" command.
2. Perhaps you want to add some content. You can add single files
with "hugo new <SECTIONNAME>\<FILENAME>.<FORMAT>".
3. Start the built-in live server via "hugo server".
Visit https://gohugo.io/ for quickstart guide and full documentation.
myblog/ 에서 적용한다.cd myblog
rm -rf public
echo "# myblog" >> README.md
git init
git add README.md
git commit -m "first commit"
git branch -M main
git remote add origin https://github.com/rain0430/myblog.git
git push -u origin main
$ git commit -m "first commit"
Author identity unknown
*** Please tell me who you are.
Run
git config --global user.email "you@example.com"
git config --global user.name "Your Name"
$ git submodule add -b main https://github.com/rain0430/rain0430.github.io.git public
$ git submodule add -f https://github.com/rain0430/rain0430.github.io.git public
$ sudo apt update
[sudo] password for evan:
Hit:1 https://artifacts.elastic.co/packages/7.x/apt stable InRelease
Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB]
Hit:3 http://archive.ubuntu.com/ubuntu focal InRelease
Get:4 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]
Get:5 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB]
Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1712 kB]
Get:7 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [916 kB]
Fetched 2963 kB in 5s (600 kB/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done
All packages are up to date.
$ sudo apt-get upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
$ sudo apt install postgresql postgresql-contrib
$ psql --version
psql (PostgreSQL) 12.9 (Ubuntu 12.9-0ubuntu0.20.04.1)
$ sudo service postgresql status
12/main (port 5432): down
$ sudo service postgresql start
* Starting PostgreSQL 12 database server
$ sudo service postgresql status
12/main (port 5432): online
$ sudo service postgresql stop
* Stopping PostgreSQL 12 database server [ OK ]
$ sudo service postgresql status
12/main (port 5432): down
$ sudo passwd postgres
New password:
Retype new password:
passwd: password updated successfully
> library(dplyr)
> library(readxl)
> raw_df = read_excel("data/NewsResult_20211213-20220313.xlsx", sheet = 1)
> raw_df2 = raw_df %>% select(일자, 언론사, 제목, URL)
> raw_df2 %>% group_by(언론사) %>% summarise(n = n())
# A tibble: 3 × 2
언론사 n
<chr> <int>
1 국민일보 180
2 조선일보 115
3 중앙일보 256
> kmib_df = raw_df2 %>% filter(언론사 == "국민일보")
> head(kmib_df, 3)
# A tibble: 3 × 4 일자 언론사 제목 URL
<chr> <chr> <chr> <chr>
1 20220312 국민일보 팬데믹의 비극 무너지는 가정, 스러지는 아이들 http://news.kmib.co…
2 20220312 국민일보 ‘장로’ 디딤돌인가 걸림돌인가 http://news.kmib.co…
3 20220311 국민일보 [기고]대전은 우리가 지킨다 http://news.kmib.co…
&cp=kd 같은 문자가 더 추가된 것을 확인할 수 있다.> kmib_df$URL[1:5]
[1] "http://news.kmib.co.kr/article/view.asp?arcid=0924235144&code=11131100"
[2] "http://news.kmib.co.kr/article/view.asp?arcid=0924235120&code=23111111"
[3] "http://news.kmib.co.kr/article/view.asp?arcid=0016856942&code=61221514&cp=kd"
[4] "http://news.kmib.co.kr/article/view.asp?arcid=0016853803&code=61111711&cp=kd"
[5] "http://news.kmib.co.kr/article/view.asp?arcid=0016847353&code=61131111&cp=kd"
> url = kmib_df$URL[1]
> news = read_html(url, encoding = "EUC-KR")
> news
{html_document}<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="ko">
[1] <head>\n<title>팬데믹의 비극… 무너지는 가정, 스러지는 아이들-국민일보</title>\n<meta http-equiv="Cont ...
[2] <body>\r\n<div id="wrap">\r\n\r\n<!-- header -->\r\n\r\n<script src="http://ww ...
위 파일을 다운로드 받은 후, “C:\mecab"에서 압축을 해제한다.
library(remotes)
install_github("junhewk/RcppMeCab")
Downloading GitHub repo junhewk/RcppMeCab@HEAD
Installing 3 packages: BH, RcppParallel, Rcpp
.
.
** testing if installed package keeps a record of temporary installation path
* DONE (RcppMeCab)
library(RcppMeCab)
library(remotes)
install_github("junhewk/RcppMeCab", force = TRUE)
library(RcppMeCab)
# 테스트
text = "안녕하세요!"
pos(sentence = text)
# $`�ȳ\xe7\xc7ϼ��\xe4!`
# [1] "�/SY" "ȳ/SL" "\xe7\xc7\xcf/SH" "���\xe4/SY"
# [5] "!/SF"
text2 = enc2utf8(text)
pos(sentence = text2)
# $`안녕하세요!`
# [1] "안녕/NNG" "하/XSV" "세요/EP+EF" "!/SF"
스타벅스 아이스 아메리카노를 선물로 보내드리겠습니다.
source 폴더 아래 images 폴더를 별도로 만든다.
# hexo logo 테스트
- 이미지
hexo server를 실행한 뒤 결과를 확인한다.
python_oracle 폴더를 생성했다.C:\Users\1\Desktop\python_oracle>
conda create --name your_env_name python=3.8
.
.
done
#
# To activate this environment, use
#
# $ conda activate python_oracle
#
# To deactivate an active environment, use
#
# $ conda deactivate
C:\Users\1\Desktop\python_oracle>conda activate python_oracle
(python_oracle) C:\Users\1\Desktop\python_oracle>
(python_oracle) C:\Users\1\Desktop\python_oracle>pip install pycaret
.
.
.
... wordcloud-1.8.1 yellowbrick-1.3.post1
(python_oracle) C:\Users\1\Desktop\python_oracle>pip install cx_Oracle
Collecting cx_Oracle
Downloading cx_Oracle-8.2.1-cp38-cp38-win_amd64.whl (219 kB)
|████████████████████████████████| 219 kB 2.2 MB/s
Installing collected packages: cx-Oracle
Successfully installed cx-Oracle-8.2.1
(python_oracle) C:\Users\1\Desktop\python_oracle> deactivate
deactivate 한 뒤, lsnrctl status 명령어를 통해 확인한다.C:\Users\1\Desktop\python_oracle>lsnrctl status
LSNRCTL for 64-bit Windows: Version 19.0.0.0.0 - Production on 16-7월 -2021 10:28:24
Copyright (c) 1991, 2019, Oracle. All rights reserved.
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=localhost)(PORT=1521)))에 연결되었습니다
리스너의 상태
------------------------
별칭 LISTENER
버전 TNSLSNR for 64-bit Windows: Version 19.0.0.0.0 - Production
시작 날짜 13-7월 -2021 17:21:03
업타임 2 일 17 시간. 7 분. 25 초
트레이스 수준 off
보안 ON: Local OS Authentication
SNMP OFF리스너 매개변수 파일 C:\ORACLE\WINDOWS.X64_193000_db_home\network\admin\listener.ora
리스너 로그 파일 C:\ORACLE\diag\tnslsnr\DESKTOP-F7LRGM5\listener\alert\log.xml
끝점 요약 청취 중...
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=1521)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=\\.\pipe\EXTPROC1521ipc)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcps)(HOST=DESKTOP-F7LRGM5)(PORT=5500))(Security=(my_wallet_directory=C:\ORACLE\admin\orcl\xdb_wallet))(Presentation=HTTP)(Session=RAW))
서비스 요약...
"CLRExtProc" 서비스는 1개의 인스턴스를 가집니다.
"CLRExtProc" 인스턴스(UNKNOWN 상태)는 이 서비스에 대해 1 처리기를 가집니다.
"orcl" 서비스는 1개의 인스턴스를 가집니다.
"orcl" 인스턴스(READY 상태)는 이 서비스에 대해 1 처리기를 가집니다.
"orclXDB" 서비스는 1개의 인스턴스를 가집니다.
"orcl" 인스턴스(READY 상태)는 이 서비스에 대해 1 처리기를 가집니다.
명령이 성공적으로 수행되었습니다
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=localhost)(PORT=1521)))에 연결되었습니다python_oracle 으로 연결이 되어 있는지 확인한다.