Brainstorming: User Data Analytics

[hollomancer]

In our database, we have a lot of open columns for stuff like branch of service, Github / LinkedIn account, that sort of thing. However, we have no way of actually populating it.

I'd like to find a way we can query the database for missing data, and start generating reports for what questions we need to ask. That way, we can do customized, targeted surveys (maybe three question a month, or every couple weeks, or something) to get a better picture of our membership base.

This all starts in the backend, but the surveys themselves would probably go out via email or Slack.

[hollomancer]

Additionally, once we get an idea of where we're missing the most data, we can start prioritizing what data is most critical (and using that to ensure we're sending targeted surveys)

[FreddosaurusRex]

hi! was just curious what the size of your data is, and whether you would want an automated report to run weekly/monthly or just a more detailed understanding of what you have now?

I haven't worked with ruby yet, but i work in the health care sector doing big data analytics/engineering (i used to be in the Air Force) and would be interested in helping out with this. I think this organization has a great mission.

[hollomancer]

Mostly, a more detailed understanding of what we have now - can we connect on Slack or email? conrad@operationcode.org

[shalombear]

Hi. I don't know Ruby but I can program in Python using requests, BeautifulSoup, and some basic NLP tools to pick up data from members' LinkedIn pages. I'm having issues using pip to install scrapy though.

I can also work on exporting this data to Excel or Access, using matplotlib and seaborn to visualize it, analyzing with pandas and sk-learn, or scripting SQL pipelines.

[apex-omontgomery]

Invited you to a channel to discuss some ideas.