MacOS M1

Airflow 활용한 API 크롤링 및 이미지 다운로드 (M1, MacOS)

개요

  • Airflow 활용해서 이미지 다운로드 받기 예제

개발환경설정

  • MacOS, M1
  • Python uv 개발환경 설정

uv 설치

curl -LsSf https://astral.sh/uv/install.sh | sh

가상환경 설정

  • 다음과 같이 설정, 프로젝트 초기화 (Python 3.11 지정)
$ uv venv -p 3.11
$ source .venv/bin/activate
  • Python 버전 확인
$ python --version 

AIRFLOW_HOME 환경변수 지정

  • airflow는 환경변수에 예민하다.
  • 프로젝트 디렉터리 경로에서 다음과 같이 추가
$ export AIRFLOW_HOME=$(pwd)/airflow
$ echo $AIRFLOW_HOME
/Users/evan/Desktop/your/project/directory/airflow

설치 스크립트 작성

  • 파일명 : install_airflow.sh
AIRFLOW_VERSION=2.8.0

# Python 버전을 3.11로 고정 설정
PYTHON_VERSION="3.11"

CONSTRAINT_URL="https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-${PYTHON_VERSION}.txt"
# For example this would install 3.0.0 with python 3.11: https://raw.githubusercontent.com/apache/airflow/constraints-3.0.0/constraints-3.11.txt

uv pip install "apache-airflow==${AIRFLOW_VERSION}" --constraint "${CONSTRAINT_URL}"
# PostgreSQL 제공자 패키지 버전 명시
uv pip install apache-airflow-providers-postgres==5.7.1
uv pip install -r requirements.txt
  • 파일명 : requirements.txt 파일 작성
pandas
numpy
seaborn
matplotlib
requests
  • sh 파일 실행 모드로 변경 후 실행
$ chmod +x install_airflow.sh
$ ./install_airflow.sh

API 확인 및 테스트

  • 다음 5개의 예정된 발사 정보 조회
$ curl "https://ll.thespacedevs.com/2.2.0/launch/upcoming/?limit=5"
{"count":338,"next":"https://ll.thespacedevs.com/2.2.0/launch/upcoming/?limit=5&offset=5","previous":null,"results":[{"id":"1a105ccb-e59f-48e8-b853-c424bd8cc699","url":"https://ll.thespacedevs.com/2.2.0/launch/1a105ccb-e59f-48e8-b853-c424bd8cc699/","slug":"falcon-9-block-5-starlink-group-6-75","name":"Falcon 9 Block 5 | Starlink Group 6-75","status":{"id":6,"name":"Launch in Flight","abbrev":"In Flight","description":"The launch vehicle has lifted off from the launchpad."},"last_updated":"2025-05-02T01:52:11Z","net":"2025-05-02T01:51:10Z","window_end":"2025-05-02T05:51:00Z","window_start":"2025-05-02T01:51:10Z","net_precision":{"id":0,"name":"Second","abbrev":"SEC","description":"The T-0 is accurate to the second."},"probability":99,"weather_concerns":null,"holdreason":"","failreason":"","hashtag":null,"launch_service_provider":{"id":121,"url":"https://ll.thespacedevs.com/2.2.0/agencies/121/","name":"SpaceX","type":"Commercial"},"rocket":{"id":8596,"configuration":{"id":164,"url":"https://ll.thespacedevs.com/2.2.0/config/launcher/164/","name":"Falcon 9","family":"Falcon","full_name":"Falcon 9 Block 5","variant":"Block 5"}},"mission":{"id":7188,"name":"Starlink Group 6-75","description":"A batch of 28 satellites for the Starlink mega-constellation - SpaceX's project for space-based Internet communication system.","launch_designator":null,"type":"Communications","orbit":{"id":8,"name":"Low Earth Orbit","abbrev":"LEO"},"agencies":[{"id":121,"url":"https://ll.thespacedevs.com/2.2.0/agencies/121/","name":"SpaceX","featured":true,"type":"Commercial","country_code":"USA","abbrev":"SpX","description":"Space Exploration Technologies Corp., known as SpaceX, is an American aerospace manufacturer and space transport services company headquartered in Hawthorne, California. It was founded in 2002 by entrepreneur Elon Musk with the goal of reducing space transportation costs and enabling the colonization of Mars. SpaceX operates from many pads, on the East Coast of the US they operate from SLC-40 at Cape Canaveral Space Force Station and historic LC-39A at Kennedy Space Center. They also operate from SLC-4E at Vandenberg Space Force Base, California, usually for polar launches. Another launch site is being developed at Boca Chica, Texas.","administrator":"CEO: Elon Musk","founding_year":"2002","launchers":"Falcon | Starship","spacecraft":"Dragon","launch_library_url":null,"total_launch_count":502,"consecutive_successful_launches":24,"successful_launches":487,"failed_launches":14,"pending_launches":118,"consecutive_successful_landings":71,"successful_landings":449,"failed_landings":26,"attempted_landings":474,"info_url":"http://www.spacex.com/","wiki_url":"http://en.wikipedia.org/wiki/SpaceX","logo_url":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/images/spacex_logo_20220826094919.png","image_url":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/images/spacex_image_20190207032501.jpeg","nation_url":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/images/spacex_nation_20230531064544.jpg"}],"info_urls":[],"vid_urls":[]},"pad":{"id":80,"url":"https://ll.thespacedevs.com/2.2.0/pad/80/","agency_id":121,"name":"Space Launch Complex 40","description":"","info_url":null,"wiki_url":"https://en.wikipedia.org/wiki/Cape_Canaveral_Air_Force_Station_Space_Launch_Complex_40","map_url":"https://www.google.com/maps?q=28.56194122,-80.57735736","latitude":"28.56194122","longitude":"-80.57735736","location":{"id":12,"url":"https://ll.thespacedevs.com/2.2.0/location/12/","name":"Cape Canaveral SFS, FL, USA","country_code":"USA","description":"Cape Canaveral Space Force Station (CCSFS) is an installation of the United States Space Force's Space Launch Delta 45, located on Cape Canaveral in Brevard County, Florida.","map_image":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/map_images/location_12_20200803142519.jpg","timezone_name":"America/New_York","total_launch_count":1020,"total_landing_count":64},"country_code":"USA","map_image":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/map_images/pad_80_20200803143323.jpg","total_launch_count":304,"orbital_launch_attempt_count":304},"webcast_live":true,"image":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/images/falcon2520925_image_20221009234147.png","infographic":null,"program":[{"id":25,"url":"https://ll.thespacedevs.com/2.2.0/program/25/","name":"Starlink","description":"Starlink is a satellite internet constellation operated by American aerospace company SpaceX","agencies":[{"id":121,"url":"https://ll.thespacedevs.com/2.2.0/agencies/121/","name":"SpaceX","type":"Commercial"}],"image_url":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/images/starlink_program_20231228154508.jpeg","start_date":"2018-02-22T14:17:00Z","end_date":null,"info_url":"https://starlink.com","wiki_url":"https://en.wikipedia.org/wiki/Starlink","mission_patches":[{"id":7,"name":"Space X Starlink Mission Patch","priority":10,"image_url":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/mission_patch_images/space2520x252_mission_patch_20221011205756.png","agency":{"id":121,"url":"https://ll.thespacedevs.com/2.2.0/agencies/121/","name":"SpaceX","type":"Commercial"}}],"type":{"id":3,"name":"Communication Constellation"}}],"orbital_launch_attempt_count":6943,"location_launch_attempt_count":1020,"pad_launch_attempt_count":304,"agency_launch_attempt_count":502,"orbital_launch_attempt_count_year":94,"location_launch_attempt_count_year":26,"pad_launch_attempt_count_year":24,"agency_launch_attempt_count_year":53,"type":"normal"},{"id":"7b685ef7-f610-413f-bd4a-cc58aed97be2","url":"https://ll.thespacedevs.com/2.2.0/launch/7b685ef7-f610-413f-bd4a-cc58aed97be2/","slug":"falcon-9-block-5-starlink-group-15-3","name":"Falcon 9 Block 5 | Starlink Group 15-3","status":{"id":8,"name":"To Be Confirmed","abbrev":"TBC","description":"Awaiting official confirmation - current date is known with some certainty."},"last_updated":"2025-04-30T19:54:30Z","net":"2025-05-03T18:13:00Z","window_end":"2025-05-03T22:13:00Z","window_start":"2025-05-03T18:13:00Z","net_precision":{"id":2,"name":"Hour","abbrev":"HR","description":"The T-0 is accurate to the hour."},"probability":null,"weather_concerns":null,"holdreason":"","failreason":"","hashtag":null,"launch_service_provider":{"id":121,"url":"https://ll.thespacedevs.com/2.2.0/agencies/121/","name":"SpaceX","type":"Commercial"},"rocket":{"id":8594,"configuration":{"id":164,"url":"https://ll.thespacedevs.com/2.2.0/config/launcher/164/","name":"Falcon 9","family":"Falcon","full_name":"Falcon 9 Block 5","variant":"Block 5"}},"mission":{"id":7186,"name":"Starlink Group 15-3","description":"A batch of 26 satellites for the Starlink mega-constellation - SpaceX's project for space-based Internet communication system.","launch_designator":null,"type":"Communications","orbit":{"id":8,"name":"Low Earth Orbit","abbrev":"LEO"},"agencies":[{"id":121,"url":"https://ll.thespacedevs.com/2.2.0/agencies/121/","name":"SpaceX","featured":true,"type":"Commercial","country_code":"USA","abbrev":"SpX","description":"Space Exploration Technologies Corp., known as SpaceX, is an American aerospace manufacturer and space transport services company headquartered in Hawthorne, California. It was founded in 2002 by entrepreneur Elon Musk with the goal of reducing space transportation costs and enabling the colonization of Mars. SpaceX operates from many pads, on the East Coast of the US they operate from SLC-40 at Cape Canaveral Space Force Station and historic LC-39A at Kennedy Space Center. They also operate from SLC-4E at Vandenberg Space Force Base, California, usually for polar launches. Another launch site is being developed at Boca Chica, Texas.","administrator":"CEO: Elon Musk","founding_year":"2002","launchers":"Falcon | Starship","spacecraft":"Dragon","launch_library_url":null,"total_launch_count":502,"consecutive_successful_launches":24,"successful_launches":487,"failed_launches":14,"pending_launches":118,"consecutive_successful_landings":71,"successful_landings":449,"failed_landings":26,"attempted_landings":474,"info_url":"http://www.spacex.com/","wiki_url":"http://en.wikipedia.org/wiki/SpaceX","logo_url":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/images/spacex_logo_20220826094919.png","image_url":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/images/spacex_image_20190207032501.jpeg","nation_url":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/images/spacex_nation_20230531064544.jpg"}],"info_urls":[],"vid_urls":[]},"pad":{"id":16,"url":"https://ll.thespacedevs.com/2.2.0/pad/16/","agency_id":null,"name":"Space Launch Complex 4E","description":"Space Launch Complex 4 East (SLC-4E) is a launch site at Vandenberg Space Force Base, California, U.S.\r\n\r\nThe pad was previously used by Atlas and Titan rockets between 1963 and 2005. The pad was built for use by Atlas-Agena rockets, but was later rebuilt to handle Titan rockets.","info_url":null,"wiki_url":"https://en.wikipedia.org/wiki/Vandenberg_Space_Launch_Complex_4#SLC-4E","map_url":"https://www.google.com/maps?q=34.632,-120.611","latitude":"34.632","longitude":"-120.611","location":{"id":11,"url":"https://ll.thespacedevs.com/2.2.0/location/11/","name":"Vandenberg SFB, CA, USA","country_code":"USA","description":"Vandenberg Space Force Base is a United States Space Force Base in Santa Barbara County, California. Established in 1941, Vandenberg Space Force Base is a space launch base, launching spacecraft from the Western Range, and also performs missile testing. The United States Space Force's Space Launch Delta 30 serves as the host delta for the base, equivalent to an Air Force air base wing. In addition to its military space launch mission, Vandenberg Space Force Base also hosts space launches for civil and commercial space entities, such as NASA and SpaceX.","map_image":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/map_images/location_11_20200803142416.jpg","timezone_name":"America/Los_Angeles","total_launch_count":804,"total_landing_count":26},"country_code":"USA","map_image":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/map_images/pad_16_20200803143532.jpg","total_launch_count":190,"orbital_launch_attempt_count":190},"webcast_live":false,"image":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/images/falcon2520925_image_20221009234147.png","infographic":null,"program":[{"id":25,"url":"https://ll.thespacedevs.com/2.2.0/program/25/","name":"Starlink","description":"Starlink is a satellite internet constellation operated by American aerospace company SpaceX","agencies":[{"id":121,"url":"https://ll.thespacedevs.com/2.2.0/agencies/121/","name":"SpaceX","type":"Commercial"}],"image_url":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/images/starlink_program_20231228154508.jpeg","start_date":"2018-02-22T14:17:00Z","end_date":null,"info_url":"https://starlink.com","wiki_url":"https://en.wikipedia.org/wiki/Starlink","mission_patches":[{"id":7,"name":"Space X Starlink Mission Patch","priority":10,"image_url":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/mission_patch_images/space2520x252_mission_patch_20221011205756.png","agency":{"id":121,"url":"https://ll.thespacedevs.com/2.2.0/agencies/121/","name":"SpaceX","type":"Commercial"}}],"type":{"id":3,"name":"Communication Constellation"}}],"orbital_launch_attempt_count":6944,"location_launch_attempt_count":805,"pad_launch_attempt_count":191,"agency_launch_attempt_count":503,"orbital_launch_attempt_count_year":95,"location_launch_attempt_count_year":19,"pad_launch_attempt_count_year":17,"agency_launch_attempt_count_year":54,"type":"normal"},{"id":"8ffb4866-43c9-46c1-aaac-05bd37891b0a","url":"https://ll.thespacedevs.com/2.2.0/launch/8ffb4866-43c9-46c1-aaac-05bd37891b0a/","slug":"falcon-9-block-5-starlink-group-6-84","name":"Falcon 9 Block 5 | Starlink Group 6-84","status":{"id":8,"name":"To Be Confirmed","abbrev":"TBC","description":"Awaiting official confirmation - current date is known with some certainty."},"last_updated":"2025-05-02T01:01:57Z","net":"2025-05-04T08:48:00Z","window_end":"2025-05-04T12:48:00Z","window_start":"2025-05-04T08:48:00Z","net_precision":{"id":2,"name":"Hour","abbrev":"HR","description":"The T-0 is accurate to the hour."},"probability":null,"weather_concerns":null,"holdreason":"","failreason":"","hashtag":null,"launch_service_provider":{"id":121,"url":"https://ll.thespacedevs.com/2.2.0/agencies/121/","name":"SpaceX","type":"Commercial"},"rocket":{"id":8598,"configuration":{"id":164,"url":"https://ll.thespacedevs.com/2.2.0/config/launcher/164/","name":"Falcon 9","family":"Falcon","full_name":"Falcon 9 Block 5","variant":"Block 5"}},"mission":{"id":7190,"name":"Starlink Group 6-84","description":"A batch of 29 satellites for the Starlink mega-constellation - SpaceX's project for space-based Internet communication system.","launch_designator":null,"type":"Communications","orbit":{"id":8,"name":"Low Earth Orbit","abbrev":"LEO"},"agencies":[{"id":121,"url":"https://ll.thespacedevs.com/2.2.0/agencies/121/","name":"SpaceX","featured":true,"type":"Commercial","country_code":"USA","abbrev":"SpX","description":"Space Exploration Technologies Corp., known as SpaceX, is an American aerospace manufacturer and space transport services company headquartered in Hawthorne, California. It was founded in 2002 by entrepreneur Elon Musk with the goal of reducing space transportation costs and enabling the colonization of Mars. SpaceX operates from many pads, on the East Coast of the US they operate from SLC-40 at Cape Canaveral Space Force Station and historic LC-39A at Kennedy Space Center. They also operate from SLC-4E at Vandenberg Space Force Base, California, usually for polar launches. Another launch site is being developed at Boca Chica, Texas.","administrator":"CEO: Elon Musk","founding_year":"2002","launchers":"Falcon | Starship","spacecraft":"Dragon","launch_library_url":null,"total_launch_count":502,"consecutive_successful_launches":24,"successful_launches":487,"failed_launches":14,"pending_launches":118,"consecutive_successful_landings":71,"successful_landings":449,"failed_landings":26,"attempted_landings":474,"info_url":"http://www.spacex.com/","wiki_url":"http://en.wikipedia.org/wiki/SpaceX","logo_url":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/images/spacex_logo_20220826094919.png","image_url":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/images/spacex_image_20190207032501.jpeg","nation_url":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/images/spacex_nation_20230531064544.jpg"}],"info_urls":[],"vid_urls":[]},"pad":{"id":87,"url":"https://ll.thespacedevs.com/2.2.0/pad/87/","agency_id":121,"name":"Launch Complex 39A","description":"","info_url":null,"wiki_url":"https://en.wikipedia.org/wiki/Kennedy_Space_Center_Launch_Complex_39#Launch_Pad_39A","map_url":"https://www.google.com/maps?q=28.60822681,-80.60428186","latitude":"28.60822681","longitude":"-80.60428186","location":{"id":27,"url":"https://ll.thespacedevs.com/2.2.0/location/27/","name":"Kennedy Space Center, FL, USA","country_code":"USA","description":"The John F. Kennedy Space Center, located on Merritt Island, Florida, is one of NASA's ten field centers. Since 1968, KSC has been NASA's primary launch center of American spaceflight, research, and technology. Launch operations for the Apollo, Skylab and Space Shuttle programs were carried out from Kennedy Space Center Launch Complex 39 and managed by KSC. Located on the east coast of Florida, KSC is adjacent to Cape Canaveral Space Force Station (CCSFS).","map_image":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/map_images/location_27_20200803142447.jpg","timezone_name":"America/New_York","total_launch_count":263,"total_landing_count":0},"country_code":"USA","map_image":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/map_images/pad_87_20200803143537.jpg","total_launch_count":205,"orbital_launch_attempt_count":204},"webcast_live":false,"image":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/images/falcon2520925_image_20221009234147.png","infographic":null,"program":[{"id":25,"url":"https://ll.thespacedevs.com/2.2.0/program/25/","name":"Starlink","description":"Starlink is a satellite internet constellation operated by American aerospace company SpaceX","agencies":[{"id":121,"url":"https://ll.thespacedevs.com/2.2.0/agencies/121/","name":"SpaceX","type":"Commercial"}],"image_url":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/images/starlink_program_20231228154508.jpeg","start_date":"2018-02-22T14:17:00Z","end_date":null,"info_url":"https://starlink.com","wiki_url":"https://en.wikipedia.org/wiki/Starlink","mission_patches":[{"id":7,"name":"Space X Starlink Mission Patch","priority":10,"image_url":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/mission_patch_images/space2520x252_mission_patch_20221011205756.png","agency":{"id":121,"url":"https://ll.thespacedevs.com/2.2.0/agencies/121/","name":"SpaceX","type":"Commercial"}}],"type":{"id":3,"name":"Communication Constellation"}}],"orbital_launch_attempt_count":6945,"location_launch_attempt_count":264,"pad_launch_attempt_count":206,"agency_launch_attempt_count":504,"orbital_launch_attempt_count_year":96,"location_launch_attempt_count_year":12,"pad_launch_attempt_count_year":12,"agency_launch_attempt_count_year":55,"type":"normal"},{"id":"82aef7fd-9664-4e94-970c-5e99eff1b331","url":"https://ll.thespacedevs.com/2.2.0/launch/82aef7fd-9664-4e94-970c-5e99eff1b331/","slug":"long-march-12-satnet-leo-group-tbd","name":"Long March 12 | SatNet LEO Group TBD?","status":{"id":1,"name":"Go for Launch","abbrev":"Go","description":"Current T-0 confirmed by official or reliable sources."},"last_updated":"2025-04-30T07:27:01Z","net":"2025-05-05T11:05:00Z","window_end":"2025-05-05T11:47:00Z","window_start":"2025-05-05T10:57:00Z","net_precision":{"id":2,"name":"Hour","abbrev":"HR","description":"The T-0 is accurate to the hour."},"probability":null,"weather_concerns":null,"holdreason":"","failreason":"","hashtag":null,"launch_service_provider":{"id":88,"url":"https://ll.thespacedevs.com/2.2.0/agencies/88/","name":"China Aerospace Science and Technology Corporation","type":"Government"},"rocket":{"id":8600,"configuration":{"id":517,"url":"https://ll.thespacedevs.com/2.2.0/config/launcher/517/","name":"Long March 12","family":"Long March","full_name":"Long March 12","variant":"12"}},"mission":{"id":7192,"name":"SatNet LEO Group TBD?","description":"A batch of Low Earth Orbit communication satellites for the Chinese state owned SatNet constellation operated by the China Satellite Network Group.\r\n\r\nThe constellation will eventually consists of 13000 satellites.","launch_designator":null,"type":"Communications","orbit":{"id":8,"name":"Low Earth Orbit","abbrev":"LEO"},"agencies":[],"info_urls":[],"vid_urls":[]},"pad":{"id":219,"url":"https://ll.thespacedevs.com/2.2.0/pad/219/","agency_id":null,"name":"Commercial LC-2","description":"","info_url":null,"wiki_url":"https://en.wikipedia.org/wiki/Wenchang_Commercial_Space_Launch_Site","map_url":"https://www.google.com/maps?q=19.59755,110.936481","latitude":"19.59755","longitude":"110.936481","location":{"id":8,"url":"https://ll.thespacedevs.com/2.2.0/location/8/","name":"Wenchang Space Launch Site, People's Republic of China","country_code":"CHN","description":"The Wenchang Space Launch Site is a rocket launch site located in Wenchang on the island of Hainan, in China.\r\n\r\nFormally a suborbital test center, it currently serves as China's southernmost spaceport. The site was selected for its low latitude, 19° north of the equator, allowing for larger payloads to be launched. It is capable of launching the Long March 5, the heaviest Chinese rocket. Unlike launch facilities on the mainland, Wenchang uses its seaport for deliveries.","map_image":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/map_images/location_8_20200803142445.jpg","timezone_name":"Asia/Shanghai","total_launch_count":38,"total_landing_count":0},"country_code":"CHN","map_image":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/map_images/pad_commercial_lc-2_20231225074048.jpg","total_launch_count":1,"orbital_launch_attempt_count":1},"webcast_live":false,"image":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/images/cz-12_on_its_la_image_20241128132937.jpg","infographic":null,"program":[],"orbital_launch_attempt_count":6946,"location_launch_attempt_count":39,"pad_launch_attempt_count":2,"agency_launch_attempt_count":521,"orbital_launch_attempt_count_year":97,"location_launch_attempt_count_year":5,"pad_launch_attempt_count_year":1,"agency_launch_attempt_count_year":20,"type":"normal"},{"id":"d5e8b971-0138-42d7-a6ba-7d43bf529d5e","url":"https://ll.thespacedevs.com/2.2.0/launch/d5e8b971-0138-42d7-a6ba-7d43bf529d5e/","slug":"falcon-9-block-5-starlink-group-6-93","name":"Falcon 9 Block 5 | Starlink Group 6-93","status":{"id":8,"name":"To Be Confirmed","abbrev":"TBC","description":"Awaiting official confirmation - current date is known with some certainty."},"last_updated":"2025-05-01T02:37:15Z","net":"2025-05-06T00:48:00Z","window_end":"2025-05-06T04:48:00Z","window_start":"2025-05-06T00:48:00Z","net_precision":{"id":2,"name":"Hour","abbrev":"HR","description":"The T-0 is accurate to the hour."},"probability":null,"weather_concerns":null,"holdreason":"","failreason":"","hashtag":null,"launch_service_provider":{"id":121,"url":"https://ll.thespacedevs.com/2.2.0/agencies/121/","name":"SpaceX","type":"Commercial"},"rocket":{"id":8599,"configuration":{"id":164,"url":"https://ll.thespacedevs.com/2.2.0/config/launcher/164/","name":"Falcon 9","family":"Falcon","full_name":"Falcon 9 Block 5","variant":"Block 5"}},"mission":{"id":7191,"name":"Starlink Group 6-93","description":"A batch of satellites for the Starlink mega-constellation - SpaceX's project for space-based Internet communication system.","launch_designator":null,"type":"Communications","orbit":{"id":8,"name":"Low Earth Orbit","abbrev":"LEO"},"agencies":[{"id":121,"url":"https://ll.thespacedevs.com/2.2.0/agencies/121/","name":"SpaceX","featured":true,"type":"Commercial","country_code":"USA","abbrev":"SpX","description":"Space Exploration Technologies Corp., known as SpaceX, is an American aerospace manufacturer and space transport services company headquartered in Hawthorne, California. It was founded in 2002 by entrepreneur Elon Musk with the goal of reducing space transportation costs and enabling the colonization of Mars. SpaceX operates from many pads, on the East Coast of the US they operate from SLC-40 at Cape Canaveral Space Force Station and historic LC-39A at Kennedy Space Center. They also operate from SLC-4E at Vandenberg Space Force Base, California, usually for polar launches. Another launch site is being developed at Boca Chica, Texas.","administrator":"CEO: Elon Musk","founding_year":"2002","launchers":"Falcon | Starship","spacecraft":"Dragon","launch_library_url":null,"total_launch_count":502,"consecutive_successful_launches":24,"successful_launches":487,"failed_launches":14,"pending_launches":118,"consecutive_successful_landings":71,"successful_landings":449,"failed_landings":26,"attempted_landings":474,"info_url":"http://www.spacex.com/","wiki_url":"http://en.wikipedia.org/wiki/SpaceX","logo_url":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/images/spacex_logo_20220826094919.png","image_url":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/images/spacex_image_20190207032501.jpeg","nation_url":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/images/spacex_nation_20230531064544.jpg"}],"info_urls":[],"vid_urls":[]},"pad":{"id":80,"url":"https://ll.thespacedevs.com/2.2.0/pad/80/","agency_id":121,"name":"Space Launch Complex 40","description":"","info_url":null,"wiki_url":"https://en.wikipedia.org/wiki/Cape_Canaveral_Air_Force_Station_Space_Launch_Complex_40","map_url":"https://www.google.com/maps?q=28.56194122,-80.57735736","latitude":"28.56194122","longitude":"-80.57735736","location":{"id":12,"url":"https://ll.thespacedevs.com/2.2.0/location/12/","name":"Cape Canaveral SFS, FL, USA","country_code":"USA","description":"Cape Canaveral Space Force Station (CCSFS) is an installation of the United States Space Force's Space Launch Delta 45, located on Cape Canaveral in Brevard County, Florida.","map_image":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/map_images/location_12_20200803142519.jpg","timezone_name":"America/New_York","total_launch_count":1020,"total_landing_count":64},"country_code":"USA","map_image":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/map_images/pad_80_20200803143323.jpg","total_launch_count":304,"orbital_launch_attempt_count":304},"webcast_live":false,"image":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/images/falcon2520925_image_20221009234147.png","infographic":null,"program":[{"id":25,"url":"https://ll.thespacedevs.com/2.2.0/program/25/","name":"Starlink","description":"Starlink is a satellite internet constellation operated by American aerospace company SpaceX","agencies":[{"id":121,"url":"https://ll.thespacedevs.com/2.2.0/agencies/121/","name":"SpaceX","type":"Commercial"}],"image_url":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/images/starlink_program_20231228154508.jpeg","start_date":"2018-02-22T14:17:00Z","end_date":null,"info_url":"https://starlink.com","wiki_url":"https://en.wikipedia.org/wiki/Starlink","mission_patches":[{"id":7,"name":"Space X Starlink Mission Patch","priority":10,"image_url":"https://thespacedevs-prod.nyc3.digitaloceanspaces.com/media/mission_patch_images/space2520x252_mission_patch_20221011205756.png","agency":{"id":121,"url":"https://ll.thespacedevs.com/2.2.0/agencies/121/","name":"SpaceX","type":"Commercial"}}],"type":{"id":3,"name":"Communication Constellation"}}],"orbital_launch_attempt_count":6947,"location_launch_attempt_count":1021,"pad_launch_attempt_count":305,"agency_launch_attempt_count":505,"orbital_launch_attempt_count_year":98,"location_launch_attempt_count_year":27,"pad_launch_attempt_count_year":25,"agency_launch_attempt_count_year":56,"type":"normal"}]}

현재까지 준비된 파일 구조

  • 파일 구조는 다음과 같다.
    • airflow 폴더 생성
    • dags 폴더 생성
    • dags 폴더 내에서 step06_rocket_image_download_filename.py 파일
tree
.
├── README.md
├── airflow
│   └── dags
│       ├── step06_rocket_image_download_filename.py
├── install_airflow.sh
└── requirements.txt

파이썬 파일 작업

  • 파일명 : step06_rocket_image_download_filename.py
  • 코드는 다음과 같다.
from datetime import datetime, timedelta
from airflow import DAG
from airflow.operators.python import PythonOperator
from airflow.operators.bash import BashOperator
import requests
import os
from urllib.parse import urlparse
import urllib.parse
import json

# macOS에서 Airflow 네트워크 요청 문제 해결
os.environ['NO_PROXY'] = '*'

# DAG의 기본 인자 설정
default_args = {
    'owner': 'airflow',
    'depends_on_past': False,
    'start_date': datetime(2024, 1, 1),
    'email_on_failure': False,
    'email_on_retry': False,
    'retries': 1,
    'retry_delay': timedelta(minutes=1),
}

def setup_airflow_home():
    """Airflow 홈 디렉토리 설정 및 환경 변수 설정"""
    try:
        # 현재 작업 디렉토리 확인
        current_dir = os.getcwd()
        # Airflow 홈 디렉토리 설정
        airflow_home = os.path.join(current_dir, 'airflow')
        # 환경 변수 설정
        os.environ['AIRFLOW_HOME'] = airflow_home
        print(f"AIRFLOW_HOME 설정됨: {airflow_home}")
        return "AIRFLOW_HOME 환경 변수 설정 완료"
    except Exception as e:
        print(f"AIRFLOW_HOME 설정 중 오류 발생: {str(e)}")
        raise

def get_launch_images():
    """Launch Library 2 API에서 로켓 발사 이미지 URL을 가져오는 함수"""
    api_url = "https://ll.thespacedevs.com/2.2.0/launch/upcoming/?limit=5"
    print(f"API 요청 시작: {api_url}")
    
    try:
        print(f"NO_PROXY 환경변수: {os.environ.get('NO_PROXY')}")
        response = requests.get(api_url, timeout=30)
        print(f"API 응답 받음: 상태 코드 {response.status_code}")
        
        if response.status_code == 200:
            data = response.json()
            launches = data['results']
            image_urls = [launch['image'] for launch in launches if launch.get('image')]
            print(f"총 {len(image_urls)}개의 이미지 URL 찾음")
            return image_urls
        else:
            raise Exception(f"API 요청 실패: {response.status_code}")
            
    except Exception as e:
        print(f"API 요청 중 오류 발생: {str(e)}")
        return [
            "https://spacelaunchnow-prod-east.nyc3.digitaloceanspaces.com/media/launch_images/falcon2520925_image_20230804070848.jpg",
            "https://spacelaunchnow-prod-east.nyc3.digitaloceanspaces.com/media/launcher_images/falcon_9_block__image_20210506060831.jpg"
        ]

def create_rocket_images_dir():
    """rocket_images 디렉토리 생성 - airflow 디렉토리 내에만 생성"""
    try:
        airflow_home = os.environ.get('AIRFLOW_HOME')
        if not airflow_home:
            raise Exception("AIRFLOW_HOME 환경변수가 설정되지 않았습니다.")
        
        # rocket_images 디렉토리 경로 설정 - airflow 디렉토리 내에 생성
        image_dir = os.path.join(airflow_home, 'rocket_images')
        os.makedirs(image_dir, exist_ok=True)
        print(f"rocket_images 디렉토리 생성됨: {image_dir}")
        return image_dir
    except Exception as e:
        print(f"디렉토리 생성 중 오류 발생: {str(e)}")
        raise

def create_output_dir():
    """output 디렉토리 생성 - airflow 디렉토리 내에만 생성"""
    try:
        airflow_home = os.environ.get('AIRFLOW_HOME')
        if not airflow_home:
            raise Exception("AIRFLOW_HOME 환경변수가 설정되지 않았습니다.")
        
        # output 디렉토리 경로 설정 - airflow 디렉토리 내에 생성
        output_dir = os.path.join(airflow_home, 'output')
        os.makedirs(output_dir, exist_ok=True)
        print(f"output 디렉토리 생성됨: {output_dir}")
        return output_dir
    except Exception as e:
        print(f"디렉토리 생성 중 오류 발생: {str(e)}")
        raise

def download_json_data():
    """Launch Library 2 API에서 JSON 데이터를 다운로드하는 함수"""
    try:
        print("JSON 데이터 다운로드 시작")
        api_url = "https://ll.thespacedevs.com/2.2.0/launch/upcoming/?limit=5"
        response = requests.get(api_url, timeout=30)
        
        if response.status_code == 200:
            data = response.json()
            
            # output 디렉토리에 JSON 파일 저장
            output_dir = create_output_dir()
            json_file_path = os.path.join(output_dir, 'launch_data.json')
            
            with open(json_file_path, 'w', encoding='utf-8') as f:
                json.dump(data, f, ensure_ascii=False, indent=4)
            
            print(f"JSON 데이터 저장 완료: {json_file_path}")
            return data
        else:
            raise Exception(f"API 요청 실패: {response.status_code}")
            
    except Exception as e:
        print(f"JSON 데이터 다운로드 중 오류 발생: {str(e)}")
        raise

def download_images():
    """로켓 발사 이미지를 다운로드하는 함수"""
    try:
        print("download_images 함수 시작")
        # JSON 데이터 가져오기
        data = download_json_data()
        launches = data['results']
        
        # AIRFLOW_HOME 환경변수 가져오기
        airflow_home = os.environ.get('AIRFLOW_HOME')
        if not airflow_home:
            raise Exception("AIRFLOW_HOME 환경변수가 설정되지 않았습니다.")
        
        # rocket_images 디렉토리 경로 설정 - airflow 디렉토리 내에 생성
        image_dir = os.path.join(airflow_home, 'rocket_images')
        os.makedirs(image_dir, exist_ok=True)
        print(f"저장 경로: {os.path.abspath(image_dir)}")
        
        downloaded_paths = []
        for idx, launch in enumerate(launches):
            try:
                if not launch.get('image'):
                    continue
                    
                url = launch['image']
                print(f"이미지 다운로드 시도 {idx+1}: {url}")
                response = requests.get(url, timeout=10)
                
                if response.status_code == 200:
                    # 파일 이름 생성 (로켓 이름과 날짜 사용)
                    rocket_name = launch.get('name', f'rocket_{idx + 1}')
                    launch_date = launch.get('net', '').split('T')[0]  # 날짜만 추출
                    safe_name = "".join(c for c in rocket_name if c.isalnum() or c in (' ', '-', '_')).rstrip()
                    file_extension = os.path.splitext(urlparse(url).path)[1] or '.jpg'
                    file_name = f'{safe_name}_{launch_date}{file_extension}'
                    file_path = os.path.join(image_dir, file_name)
                    
                    with open(file_path, 'wb') as f:
                        f.write(response.content)
                    downloaded_paths.append(file_path)
                    print(f"이미지 저장 성공: {file_path}")
                else:
                    print(f"이미지 다운로드 실패 - 상태 코드: {response.status_code}")
            
            except Exception as e:
                print(f"개별 이미지 다운로드 실패 (URL: {url}): {str(e)}")
        
        return f"총 {len(downloaded_paths)}개의 이미지 다운로드 완료"
        
    except Exception as e:
        print(f"전체 프로세스 실패: {str(e)}")
        return f"오류 발생: {str(e)}"

# DAG 정의
dag = DAG(
    'step06_rocket_image_download',
    default_args=default_args,
    description='로켓 발사 이미지 다운로드',
    schedule_interval=timedelta(days=1),
    catchup=False
)

# AIRFLOW_HOME 설정 태스크
setup_env_task = PythonOperator(
    task_id='setup_airflow_home',
    python_callable=setup_airflow_home,
    dag=dag
)

# Hello Airflow 태스크
hello_task = BashOperator(
    task_id='hello_task',
    bash_command='echo "Hello Airflow" && echo "AIRFLOW_HOME: $AIRFLOW_HOME"',
    dag=dag
)

# 디렉토리 생성 태스크
create_dir_task = PythonOperator(
    task_id='create_dir_task',
    python_callable=create_rocket_images_dir,
    dag=dag
)

# JSON 데이터 다운로드 태스크
download_json_task = PythonOperator(
    task_id='download_json_data',
    python_callable=download_json_data,
    dag=dag
)

# 이미지 다운로드 태스크
download_task = PythonOperator(
    task_id='download_rocket_images',
    python_callable=download_images,
    dag=dag
)

# Task 의존성 설정
setup_env_task >> hello_task >> create_dir_task >> download_json_task >> download_task

테스트

  • 파일 작성이 완료가 되면 Airflow 실행을 할 것이다.
$ airflow db reset -y
$ airflow db init
  • 사용자 생성
$ airflow users create \
    --username admin \
    --firstname admin \
    --lastname admin \
    --role Admin \
    --email admin@example.com \
    --password 1234
  • 웹서버 실행
$ airflow webserver -p 8080
  • 다른 bash 터미널 열고 진행
$ export AIRFLOW_HOME=$(pwd)/airflow
$ airflow scheduler

Screenshot 2025-05-02 at 11.49.42 AM.png

Oracle 19c Installation Using Docker on M1

개요

  • Docker 활용해서 Oracle 설치하기 on M1
  • SQL Developer 활용해서 접속 및 테스트하기

사전조건

  • Docker가 이미 설치가 되어 있다고 가정한다.

Oracle Database 설치 순서

Step 01 : 도커 이미지 가져오기

  • Clone Oracle’s Docker Images Repository, Open your terminal and run
git clone https://github.com/oracle/docker-images

Step 02 : Oracle Database 19c 파일 다운로드

Screenshot 2025-03-15 at 10.17.41 AM.png

Ubuntu install on M1

개요

  • M1에서 Ubuntu를 설치하는 방법에 대해 기술한다.

Ubuntu 24.04 LTS 다운로드

  • Ubuntu Download를 진행한다.
    • 다운로드 받을 시, arm으로 다운로드 받아야 한다.
    • 다른 아키텍처로 다운로드 받을 시 리눅스가 활성화가 되지 않는다.
  • 사이트 : https://ubuntu.com/download/server/arm

Screenshot 2024-05-21 at 9.43.12 AM.png

Screenshot 2024-05-21 at 10.46.43 AM.png

UTM 다운로드

Screenshot 2024-05-21 at 9.29.15 AM.png

UTM 가상머신 생성

  • UTM을 실행하면 아래와 같은 화면이 나온다.
  • Create a New Virtual Machine을 선택한다.

Screenshot 2024-05-21 at 9.30.52 AM.png

MySQL 삭제, 재설치 가이드 on M1

개요

  • M1에서 MySQL을 설치 하고 Workbench에 접속하는 과정을 설명한다.
  • 데이터 로드 시, ASCII 에러 과정 해결하는 방법도 살펴본다. (임시방편)

사전학습

  • brew 명령어를 알고 있는 분에 한해 작성을 하였다.

주의

  • 아래 코드 복사할 시, $ 는 제외 후 복사한다.

MySQL 실행 확인 후 프로세스 Kill

  • 먼저 MySQL이 실행중인지를 확인한다.
$ brew services list
Name  Status  User File
mysql started evan ~/Library/LaunchAgents/homebrew.mxcl.mysql.plist
  • 서비스를 강제 종료한다.
$ brew services stop mysql
Stopping `mysql`... (might take a while)
==> Successfully stopped `mysql` (label: homebrew.mxcl.mysql)

관련 파일 삭제

  • 재 설치를 위해서는 기존에 설치된 파일 목록 등을 모두 제거한다.
$ which mysql
/opt/homebrew/bin/mysql
  • Homebrew로 기존에 설치했다면 아래 명령어를 실행한다.
$ brew uninstall --force mysql
Uninstalling mysql... (323 files, 312.8MB)
  • 다음 라인을 한줄씩 실행한다.
sudo rm -rf /usr/local/mysql
sudo rm -rf /usr/local/bin/mysql
sudo rm -rf /usr/local/var/mysql
sudo rm -rf /usr/local/Cellar/mysql
sudo rm -rf /usr/local/mysql*
sudo rm -rf /tmp/mysql.sock.lock
sudo rm -rf /tmp/mysqlx.sock.lock
sudo rm -rf /tmp/mysql.sock
sudo rm -rf /tmp/mysqlx.sock
sudo rm ~/Library/LaunchAgents/homebrew.mxcl.mysql.plist
sudo rm -rf /Library/StartupItems/MySQLCOM
sudo rm -rf /Library/PreferencePanes/My*

MySQLWorkbench 삭제

  • Applications 폴더에서 해당 MySQLWorkbench 파일을 찾아 삭제한다.

Screenshot 2024-04-02 at 1.20.31 PM.png

MySQL Workbench File Import Error in Mac

개요

  • MySQL Workbench에서 File을 불러올 때 에러가 발생했을 때 대처 요령을 소개한다.
  • Workbench에서 File을 불러올 때 가끔 아래와 같은 아래가 발생하곤 한다.

Screenshot 2023-10-04 at 11.33.28 AM.png

  • 해결방법은 MySQL Shell Script에서 직접 파일을 불러오는 방식이다.

MySQL 재접속

  • 기존에 Workbench에 접속해 있었다면 우선 종료를 한다.

Screenshot 2023-10-04 at 11.35.16 AM.png

  • 재접속 전 Edit Connection 버튼을 클릭한다.

    Screenshot 2023-10-04 at 11.36.03 AM.png

  • Advanced Tab을 클릭한다.

    • Others 메뉴에서 OPT_LOCAL_INFILE=1 을 입력한다.
    • Test Connection 버튼을 클릭하여 정상적으로 접속이 되는지 재 확인한다.

    Screenshot 2023-10-04 at 11.36.43 AM.png

M1 환경설정 XGBoost & LightGBM with Streamlit in Python

개요

  • M1에서 Python 환경설정을 해본다.
  • XGBoost & LightGBM 및 Streamlit 설치를 진행한다.

아나콘다 설치

  • m1 버전의 아나콘다를 설치한다.

fig01.png

Screenshot 2023-09-02 at 12.06.58 AM.png

깃헙 레포 생성

  • 먼저 github repo를 생성한다.

Screenshot 2023-09-02 at 12.14.00 AM.png

Conda 가상환경 설정

  • git clone 명령어를 통해 repo를 로컬로 다운로드 한다.
evan$ git clone https://github.com/yourname/m1_streamlit.git
Cloning into 'm1_streamlit'...
remote: Enumerating objects: 4, done.
remote: Counting objects: 100% (4/4), done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 4 (delta 0), reused 0 (delta 0), pack-reused 0
Receiving objects: 100% (4/4), done.
  • conda 명령어를 통해 가상환경을 설치한다.
  • 그 후에 가상환경에 접속 후, Python 설치 및 주요 라이브러리를 설치한다.
  • 설치 중간에 메시지가 나오면 Y를 입력 후 계속 설치 한다.
evan$ cd m1_streamlit
m1_streamlit evan$ conda create -n mulcamp python=3.10.3
m1_streamlit evan$ conda activate mulcamp
(mulcamp) m1_streamlit evan$ conda install numpy scipy scikit-learn pandas

Screenshot 2023-09-02 at 12.21.46 AM.png

M1 Mac Tensorflow Installation in R

개요

  • M1 Mac에서 텐서플로를 설치 한다.
  • 필자의 현재 M1 환경은 아래와 같다.
sessionInfo()
R version 4.1.2 (2021-11-01)
Platform: aarch64-apple-darwin20 (64-bit)
Running under: macOS Big Sur 11.6

Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] ggplot2_3.3.5    dplyr_1.0.7      tfdatasets_2.7.0 keras_2.7.0     
[5] reticulate_1.22  tensorflow_2.7.0

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.7        compiler_4.1.2    pillar_1.6.4      prettyunits_1.1.1
 [5] base64enc_0.1-3   tools_4.1.2       progress_1.2.2    digest_0.6.28    
 [9] zeallot_0.1.0     nlme_3.1-153      gtable_0.3.0      jsonlite_1.7.2   
[13] lifecycle_1.0.1   tibble_3.1.6      lattice_0.20-45   mgcv_1.8-38      
[17] pkgconfig_2.0.3   png_0.1-7         rlang_0.4.12      Matrix_1.3-4     
[21] cli_3.1.0         rstudioapi_0.13   withr_2.4.2       generics_0.1.1   
[25] vctrs_0.3.8       hms_1.1.1         rprojroot_2.0.2   grid_4.1.2       
[29] tidyselect_1.1.1  glue_1.5.0        here_1.0.1        R6_2.5.1         
[33] fansi_0.5.0       farver_2.1.0      purrr_0.3.4       magrittr_2.0.1   
[37] whisker_0.4       splines_4.1.2     scales_1.1.1      tfruns_1.5.0     
[41] ellipsis_0.3.2    colorspace_2.0-2  labeling_0.4.2    utf8_1.2.2       
[45] munsell_0.5.0     crayon_1.4.2 

Miniforge3 설치

Python과 Oracle 연동

개요

  • 파이참에서 가상환경을 만들어 오라클 연동 예제를 작성한다.
  • 아나콘다, 파이참, 그리고 오라클 설치는 생략한다.

1. 가상환경 활성화

  • cmd 창에서 가상 환경을 세팅 하도록 한다. (권장: 관리자 실행)
  • 바탕화면에 필자는 python_oracle 폴더를 생성했다.
  • 현재 경로는 아래와 같다.
C:\Users\1\Desktop\python_oracle> 
  • 먼저 가상환경을 만든다.
conda create --name your_env_name python=3.8
.
.
done
#
# To activate this environment, use
#
#     $ conda activate python_oracle
#
# To deactivate an active environment, use
#
#     $ conda deactivate
  • your_env_name 대신 다른 이름으로 설정해도 된다.
  • 가상 환경에 접속한다.

C:\Users\1\Desktop\python_oracle>conda activate python_oracle
(python_oracle) C:\Users\1\Desktop\python_oracle>

2. 필수 라이브러리 설치

  • ML을 위한 필수 라이브러리를 설치한다.
  • pycaret & oracle
(python_oracle) C:\Users\1\Desktop\python_oracle>pip install pycaret
.
.
.
... wordcloud-1.8.1 yellowbrick-1.3.post1
(python_oracle) C:\Users\1\Desktop\python_oracle>pip install cx_Oracle
Collecting cx_Oracle
  Downloading cx_Oracle-8.2.1-cp38-cp38-win_amd64.whl (219 kB)
     |████████████████████████████████| 219 kB 2.2 MB/s
Installing collected packages: cx-Oracle
Successfully installed cx-Oracle-8.2.1
(python_oracle) C:\Users\1\Desktop\python_oracle> deactivate
  • deactivate 한 뒤, lsnrctl status 명령어를 통해 확인한다.
C:\Users\1\Desktop\python_oracle>lsnrctl status
LSNRCTL for 64-bit Windows: Version 19.0.0.0.0 - Production on 16-7월 -2021 10:28:24

Copyright (c) 1991, 2019, Oracle.  All rights reserved.

(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=localhost)(PORT=1521)))에 연결되었습니다
리스너의 상태
------------------------
별칭                     LISTENER
버전                     TNSLSNR for 64-bit Windows: Version 19.0.0.0.0 - Production
시작 날짜                 13-7월 -2021 17:21:03
업타임                   217 시간. 7 분. 25트레이스 수준            off
보안                     ON: Local OS Authentication
SNMP                     OFF리스너 매개변수 파일   C:\ORACLE\WINDOWS.X64_193000_db_home\network\admin\listener.ora
리스너 로그 파일         C:\ORACLE\diag\tnslsnr\DESKTOP-F7LRGM5\listener\alert\log.xml
끝점 요약 청취 중...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=1521)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=\\.\pipe\EXTPROC1521ipc)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcps)(HOST=DESKTOP-F7LRGM5)(PORT=5500))(Security=(my_wallet_directory=C:\ORACLE\admin\orcl\xdb_wallet))(Presentation=HTTP)(Session=RAW))
서비스 요약...
"CLRExtProc" 서비스는 1개의 인스턴스를 가집니다.
  "CLRExtProc" 인스턴스(UNKNOWN 상태)는 이 서비스에 대해 1 처리기를 가집니다.
"orcl" 서비스는 1개의 인스턴스를 가집니다.
  "orcl" 인스턴스(READY 상태)는 이 서비스에 대해 1 처리기를 가집니다.
"orclXDB" 서비스는 1개의 인스턴스를 가집니다.
  "orcl" 인스턴스(READY 상태)는 이 서비스에 대해 1 처리기를 가집니다.
명령이 성공적으로 수행되었습니다
  • (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=localhost)(PORT=1521)))에 연결되었습니다

3. 연동코드 작성

  • 필수 정보를 확인한다.
    • IP주소: localhost
    • Port번호: 1521
    • 서비스이름: orcl
  • JupyterLab을 연 후 코드를 작성한다. (관리자 살행)
    • 가상환경 python_oracle 으로 연결이 되어 있는지 확인한다.

PostgreSQL 기초 1

개요

  • psql shell 명령어를 간단히 배우도록 한다.
  • Database, Schema, Table를 생성하도록 한다.
  • 데이터타입에 대해 배우도록 한다.

psql Shell 명령어

명령어 설명
\q psql을 종료한다.
\l 데이터베이스를 조회한다.
\c 입력한 DB로 이동한다.
\e 외부편집기로 sql 쿼리 입력 가능
\dt 현재 DB에서 테이블을 확인할 수 있음.
  • 자주 사용하는 명령어이기 때문에 확인한다.

DB 및 Table 다루기

  • 콘솔창에서 book 이름의 DB를 생성한다.
evan=# CREATE DATABASE book;
CREATE DATABASE
  • 그 후, book에 접속한다.
evan=# \c book
You are now connected to database "book" as user "evan".
book=#
  • 이번에는 DROP DATABASE 구문을 통해 book 삭제한다.
evan=# DROP DATABASE book;
DROP DATABASE

SCHEMA 생성

postgres=# CREATE SCHEMA ann_simmons;
CREATE SCHEMA

postgres=# CREATE SCHEMA ty_beck;
CREATE SCHEMA

postgres=# CREATE SCHEMA production;
CREATE SCHEMA

Two Table 생성

  • 두개의 테이블을 생성한다.
CREATE TABLE customers ();
CREATE TABLE orders ();
  • 커밋을 항상 진행한다.
  • psql 터미널에서 \dt를 실행하여 출력값이 잘 나오는지 확인한다.
postgres=# \dt
         List of relations
 Schema |   Name    | Type  | Owner 
--------+-----------+-------+-------
 public | customers | table | evan
 public | orders    | table | evan
(2 rows)

Primary Key 생성

  • 이번에는 Primary Key 생성하는 것을 준비한다.
  • 이제 이 두개의 Table를 생성하도록 합니다.
CREATE TABLE business_type (
	id serial PRIMARY KEY,
  	description TEXT NOT NULL
);

CREATE TABLE applicant (
	id serial PRIMARY KEY,
  	name TEXT NOT NULL,
  	zip_code CHAR(5) NOT NULL,
  	business_type_id INTEGER references business_type(id)
);

CREATE TABLE users (
  id serial PRIMARY KEY,
  first_name TEXT NOT NULL,
  last_name TEXT NOT NULL,
  email TEXT NOT NULL,
  hashed_password CHAR(72) NOT NULL
);

schema.table 생성

  • temp schema를 생성한 후, temp.users table를 작성한다.
CREATE TABLE temp.users (
  id serial PRIMARY KEY,
  first_name TEXT NOT NULL,
  last_name TEXT NOT NULL,
  email TEXT NOT NULL,
  hashed_password CHAR(72) NOT NULL
);

Data Types

  • 크게 Text, NUMERIC, BOOLEAN 형태로 구성이 됩니다.
-- Create the project table
CREATE TABLE project (
	-- Primary Key
	id SERIAL PRIMARY KEY,
    -- franchise 기회
	is_franchise BOOLEAN DEFAULT FALSE,
	-- Franchise 이름
    franchise_name TEXT DEFAULT NULL,
    -- Project 시도
    project_state TEXT,
    -- Project 시군
    project_county TEXT,
    -- Project로 발생한 Job 수
    jobs_supported NUMERIC
);

다양한 Text Data Types

CREATE TABLE book (
  isbn CHAR(13) NOT NULL, 
  author_first_name VARCHAR(50) NOT NULL, 
  author_last_name VARCHAR(50) NOT NULL, 
  content TEXT NOT NULL
);
  • Text data 타입은: TEXT, VARCHAR(N), CHAR(N) 크게 3가지 형태로 구성이 된다.
  • TEXT
    • 길이를 알 수 없는 텍스트 데이터에 적합
  • VARCHAR(N)
    • N은 최대 길이가 N자인 문자열만 저장하도록 열을 제한함.
    • N보다 글자수가 큰 값이 입력이 되면 error가 발생이 됨
    • 만약 N을 지정하지 않으면 TEXT와 동일하게 됨
  • CHAR(N)
    • 정확하게 N 숫자만큼 글자만 들어가게 됨
    • isbn은 주로 출판사에서 볼 수 있는 문자열임

Numeric Data

CREATE TABLE employee (
  id SERIAL PRIMARY KEY, 
  first_name VARCHAR(10) NOT NULL, 
  last_name VARCHAR(10) NOT NULL, 
  num_sales INTEGER
)

-- Create the client table
CREATE TABLE client (
  id SERIAL PRIMARY KEY,
  name VARCHAR(50),
  site_url VARCHAR(50),
  num_employees SMALLINT,
  num_customers INTEGER
);

-- Create the campaign table
CREATE TABLE campaign (
  id SERIAL PRIMARY KEY,
  name VARCHAR(50),
  budget NUMERIC(7, 2),
  num_days SMALLINT DEFAULT 30,
  goal_amount INTEGER DEFAULT 100,
  num_applications INTEGER DEFAULT 0
);
  • Integer Types

PostgreSQL 테이블 생성 예제

테이블 생성 및 수정 삭제

  • pgAdmin4을 활용한 테이블 수정 삭제
  • Schemas에서 public-Table 마우스 오른쪽 버튼을 누른 뒤 Query Tool을 선택합니다.

  • developers 테이블을 별도로 생성합니다.
CREATE TABLE developers (
	book_id INTEGER, 
	date DATE,
	name VARCHAR(80)
)
CREATE TABLE
Query returned successfully in 65 msec.
  • 이번에는 값을 입력하도록 합니다.
INSERT INTO developers VALUES(1, '2019-12-17', '"자바"')
  • 그런데, 작은 따옴표(’)를 넣고 싶을 때는 큰 따옴표(")로 깜사면, 에러가 발생이 됩니다.
INSERT INTO developers VALUES(2, '2019-12-17', "'자바'")
ERROR: 오류:  "'자바'" 이름의 칼럼은 없습니다
LINE 1: INSERT INTO developers VALUES(1, '2019-12-17', "'자바'")
                                                       ^
SQL state: 42703
Character: 48
  • 이런 경우에는 작은 따옴표를 두번 입혀서 깜사도록 합니다.
INSERT INTO developers VALUES(2, '2019-12-17', '''자바''')
INSERT 0 1

Query returned successfully in 38 msec.
  • 이번에는 Python’s Programming을 입력하도록 합니다.
  • 입력하고자 하는 ‘주위로 각각 두번 ’’ 입력하도록 합니다.
INSERT INTO developers VALUES(3, '2019-12-17', 'Python''s Programming');
INSERT 0 1

Query returned successfully in 32 msec.

테이블 조회

  • 일단 먼저 잘 조회가 되는지 확인 하도록 합니다.
SELECT * FROM developers;