Data Architect & Business Strategist (CSV Audit & Pipeline)
Contributed by somebeing2
Improved by Laravel Company · 2026-09-07
Improved prompt:
I require you to assume the role of a Senior Data Science Architect and Lead Business Analyst. I am uploading a CSV file containing raw data that I need processed. Your primary objective is to conduct a thorough technical audit and develop a production-grade data cleaning pipeline that directly aligns with the business's strategic objectives.
Please adhere to the following 4-step execution workflow:
Technical Audit & Business Context: Thoroughly analyze the schema of the uploaded data. Identify and catalog all inconsistencies, missing values, and potential "data smells" such as outliers, duplicates, or anomalous entries. In your analysis, explicitly explain how these identified data issues could directly impact the accuracy and reliability of business decision-making processes. Provide concrete examples, such as how inconsistent date formats might lead to inaccuracies in month-over-month sales trend analysis or how missing customer segments could skew marketing effectiveness metrics.
Statistical Strategy: Based on the audit findings, propose and justify a comprehensive statistical strategy for data imputation, encoding, and scaling techniques. Clearly differentiate between using mean imputation for continuous variables and median imputation for ordinal variables. Explain the trade-offs between one-hot encoding and label encoding for categorical features, and provide the rationale for choosing one over the other based on the specific dataset characteristics. Similarly, discuss the nuances between standard scaling and robust scaling, and recommend the most suitable approach for this particular data set.
The Implementation Block: Write a modular, clean, and well-commented Python script using the pandas and scikit-learn libraries. Ensure the script adheres to the PEP8 style guide for Python code. Implement a Pipeline object using the Pipeline API from scikit-learn so the code is ready for integration into a Streamlit dashboard for interactive exploration or an automated batch job for regular data processing. Include all necessary data transformations within the pipeline stages.
Post-Processing Validation: Develop a series of assertive checks to ensure the integrity and quality of the processed data. Implement checks to verify the complete absence of null values in the final data frame, and add assertions for data types and memory optimization using appropriate dtypes such as int8 or float32 to maximize efficiency. If a target variable is present in the data, ensure there is no data leakage that could compromise the model's performance in an unbiased manner.
Constraints:
Place a strong emphasis on optimizing memory usage by selecting appropriate dtypes for the columns.
Ensure that the data pipeline is designed to handle potential future data growth without significant performance degradation.
Provide the output in a structured Markdown format with professional code comments explaining the purpose of each section and line of code for enhanced readability and maintainability.
I have attached the CSV file for analysis. Please commence the audit process.
Original prompt (before our improvements)
I want you to act as a Senior Data Science Architect and Lead Business Analyst. I am uploading a CSV file that contains raw data. Your goal is to perform a deep technical audit and provide a production-ready cleaning pipeline that aligns with business objectives. Please follow this 4-step execution flow: Technical Audit & Business Context: Analyze the schema. Identify inconsistencies, missing values, and Data Smells. Briefly explain how these data issues might impact business decision-making (e.g., Inconsistent dates may lead to incorrect monthly trend analysis). Statistical Strategy: Propose a rigorous strategy for Imputation (Median vs. Mean), Encoding (One-Hot vs. Label), and Scaling (Standard vs. Robust) based on the audit. The Implementation Block: Write a modular, PEP8-compliant Python script using pandas and scikit-learn. Include a Pipeline object so the code is ready for a Streamlit dashboard or an automated batch job. Post-Processing Validation: Provide assertion checks to verify data integrity (e.g., checking for nulls or memory optimization via down casting). Constraints: Prioritize memory efficiency (use appropriate dtypes like int8 or float32). Ensure zero data leakage if a target variable is present. Provide the output in structured Markdown with professional code comments. I have uploaded the file. Please begin the audit.