Showing posts with label model. Show all posts
Showing posts with label model. Show all posts

Monday, March 12, 2012

Question on Microsoft Association Rules

I build a data mining model to predict what are the best studying methods for the student to pass the examinaton.

Create Mining Model StudentAssociation (
Student_No long key,
Gender text discrete predict,
PassOrFail text discrete predict,
StudyMethod table predict ( MethodName text key )
) Using Microsoft_Association_Rules ( Minimum_Support=0.02, Minimum_Probability=0.03 )

The mining table will contain all the methods that the students use, no matter their examination is passed or failed.
The value of PassOrFail will have either 'Pass' or 'Fail'.

According to the above model, can I query the best studying methods?
Or I should only train the model with the student who pass the examination, and ignore all the failed.

Thanks.
Joe.

I wouldn't use AR, try Decision Trees, Naive Bayes, or Neural Nets. Also, if you're trying to predict what study methods indicate pass or fail, you only need to make PassOrFail predictable, not the nested table.

You definately need positive and negative examples to predict the result. You could create a clustering model on only the passing students to see which types of study methods group together for passing students.

|||

As your suggestion to create a clustering model on only the passing students, then I suppose to find that student uses both "method A" and "method B" will have a highest chance to pass the examination.
However, "Method A" and "Method B" may also have the highest probability to fail if I study the failed students. Am I right?
Do I need to create two clustering models, one for passing and one for fail, in order to have a complete picture?
Or, is there any methods to accomplish this task?

Thanks Jamie.

|||

Actually, if you only use passing students, then you won't see that Method A and Method B have higher chances than any other method, since all students passed. What you will see is what methods are used together by passing students. For example, there may be a group of passing students who use methods A and B, another using methods C and D, and yet another using A and D.

You are right in that these methods could just as easily be used by failing students. You should also create a similar model for failing students. You may find that the clusters are the same, they may be different, they may be similar with different proportions. For example, you may have an A and B cluster in both models, but in the passing model you find 40% of the students in A and B and only 5% in the failing model.

Another option if you want to use clustering is to make the Pass/Fail column "Predict Only" - in this case the algorithm will cluster based on all other attributes ignoring pass/fail, and then apply statistics for pass/fail across the clusters that were created. This will show if pass/fail is independent of method groupings or not (you can use the cluster diagram for this). However, if you want specifically to predict pass/fail based on study methods, you are better off using Trees or Neural Nets.

|||

Your answer is very useful for me.

Thanks Jamie.

Saturday, February 25, 2012

Question on column mappings between mining structure and case table for lift chart

Hi, all experts here,

I am a bit confused for the model evaluation (lift chart), should we map all the columns for both the mining structure and the case table? I mean for those predictive models, we have a predict column, shouldnt we ignore the mapping of the predictive column between the mining structure and the case table? But it seemes we are not allowed to miss the predictive column mapping between the mining structure and the case table.

Why is that? Could any experts here give me some explanation on that?

Hope my question is clear for your help.

Thanks a lot and I am looking forward to hearing from you shortly.

With best regards,

Yours sincerely,

The mapping of the predicted column is required to compare the prediction with the actual test data. It is not used in the actual prediction|||

Hi, Bogdan,

Thanks a lot for your advices.

With best regards,

Yours sincerely,

question on changing recovery model on DB

Hi,
I've got several production DBs that I want to change the recovery model
on from simple to full. Are there any caveats I need to be aware of in
making this change? Or is it just a matter of selecting the 'Full' recovery
model under the 'Options' tab of the DBs Properties and hitting the 'OK'
button? Thanks.
Make sure you set up log backup jobs for the databases to prevent the log
file sizes getting out of hand.
HTH
Jasper Smith (SQL Server MVP)
http://www.sqldbatips.com
I support PASS - the definitive, global
community for SQL Server professionals -
http://www.sqlpass.org
"mb" <mb@.discussions.microsoft.com> wrote in message
news:2547FAF9-8BB6-407C-AFCA-E5D2F9082EB0@.microsoft.com...
> Hi,
> I've got several production DBs that I want to change the recovery model
> on from simple to full. Are there any caveats I need to be aware of in
> making this change? Or is it just a matter of selecting the 'Full'
> recovery
> model under the 'Options' tab of the DBs Properties and hitting the 'OK'
> button? Thanks.
|||mb wrote:
> Hi,
> I've got several production DBs that I want to change the recovery
> model on from simple to full. Are there any caveats I need to be
> aware of in making this change? Or is it just a matter of selecting
> the 'Full' recovery model under the 'Options' tab of the DBs
> Properties and hitting the 'OK' button? Thanks.
You must immediately perform a Full Backup or your t-logs will still get
truncated every checkpoint.
David Gugick
Quest Software
www.imceda.com
www.quest.com
|||Hi,
Once you change your database from SIMPLE to FULL recovery model, you need
to recreate your backup chain by doing:-
1. A Full databaase backup
2. Schedule a transaction log backup frequently
In FULL recovery you need to perform the log backup, this will be helpful to
recover the database point-in-time as
well as help you to keep your LDF file size in control.
Have a look into Recovery model topic in books online for more detail.
Thanks
Hari
SQL SERVER MVP
"mb" <mb@.discussions.microsoft.com> wrote in message
news:2547FAF9-8BB6-407C-AFCA-E5D2F9082EB0@.microsoft.com...
> Hi,
> I've got several production DBs that I want to change the recovery model
> on from simple to full. Are there any caveats I need to be aware of in
> making this change? Or is it just a matter of selecting the 'Full'
> recovery
> model under the 'Options' tab of the DBs Properties and hitting the 'OK'
> button? Thanks.

question on changing recovery model on DB

Hi,
I've got several production DBs that I want to change the recovery model
on from simple to full. Are there any caveats I need to be aware of in
making this change? Or is it just a matter of selecting the 'Full' recovery
model under the 'Options' tab of the DBs Properties and hitting the 'OK'
button? Thanks.Make sure you set up log backup jobs for the databases to prevent the log
file sizes getting out of hand.
--
HTH
Jasper Smith (SQL Server MVP)
http://www.sqldbatips.com
I support PASS - the definitive, global
community for SQL Server professionals -
http://www.sqlpass.org
"mb" <mb@.discussions.microsoft.com> wrote in message
news:2547FAF9-8BB6-407C-AFCA-E5D2F9082EB0@.microsoft.com...
> Hi,
> I've got several production DBs that I want to change the recovery model
> on from simple to full. Are there any caveats I need to be aware of in
> making this change? Or is it just a matter of selecting the 'Full'
> recovery
> model under the 'Options' tab of the DBs Properties and hitting the 'OK'
> button? Thanks.|||mb wrote:
> Hi,
> I've got several production DBs that I want to change the recovery
> model on from simple to full. Are there any caveats I need to be
> aware of in making this change? Or is it just a matter of selecting
> the 'Full' recovery model under the 'Options' tab of the DBs
> Properties and hitting the 'OK' button? Thanks.
You must immediately perform a Full Backup or your t-logs will still get
truncated every checkpoint.
--
David Gugick
Quest Software
www.imceda.com
www.quest.com|||Hi,
Once you change your database from SIMPLE to FULL recovery model, you need
to recreate your backup chain by doing:-
1. A Full databaase backup
2. Schedule a transaction log backup frequently
In FULL recovery you need to perform the log backup, this will be helpful to
recover the database point-in-time as
well as help you to keep your LDF file size in control.
Have a look into Recovery model topic in books online for more detail.
Thanks
Hari
SQL SERVER MVP
"mb" <mb@.discussions.microsoft.com> wrote in message
news:2547FAF9-8BB6-407C-AFCA-E5D2F9082EB0@.microsoft.com...
> Hi,
> I've got several production DBs that I want to change the recovery model
> on from simple to full. Are there any caveats I need to be aware of in
> making this change? Or is it just a matter of selecting the 'Full'
> recovery
> model under the 'Options' tab of the DBs Properties and hitting the 'OK'
> button? Thanks.

question on changing recovery model on DB

Hi,
I've got several production DBs that I want to change the recovery model
on from simple to full. Are there any caveats I need to be aware of in
making this change? Or is it just a matter of selecting the 'Full' recovery
model under the 'Options' tab of the DBs Properties and hitting the 'OK'
button? Thanks.Make sure you set up log backup jobs for the databases to prevent the log
file sizes getting out of hand.
HTH
Jasper Smith (SQL Server MVP)
http://www.sqldbatips.com
I support PASS - the definitive, global
community for SQL Server professionals -
http://www.sqlpass.org
"mb" <mb@.discussions.microsoft.com> wrote in message
news:2547FAF9-8BB6-407C-AFCA-E5D2F9082EB0@.microsoft.com...
> Hi,
> I've got several production DBs that I want to change the recovery model
> on from simple to full. Are there any caveats I need to be aware of in
> making this change? Or is it just a matter of selecting the 'Full'
> recovery
> model under the 'Options' tab of the DBs Properties and hitting the 'OK'
> button? Thanks.|||mb wrote:
> Hi,
> I've got several production DBs that I want to change the recovery
> model on from simple to full. Are there any caveats I need to be
> aware of in making this change? Or is it just a matter of selecting
> the 'Full' recovery model under the 'Options' tab of the DBs
> Properties and hitting the 'OK' button? Thanks.
You must immediately perform a Full Backup or your t-logs will still get
truncated every checkpoint.
David Gugick
Quest Software
www.imceda.com
www.quest.com|||Hi,
Once you change your database from SIMPLE to FULL recovery model, you need
to recreate your backup chain by doing:-
1. A Full databaase backup
2. Schedule a transaction log backup frequently
In FULL recovery you need to perform the log backup, this will be helpful to
recover the database point-in-time as
well as help you to keep your LDF file size in control.
Have a look into Recovery model topic in books online for more detail.
Thanks
Hari
SQL SERVER MVP
"mb" <mb@.discussions.microsoft.com> wrote in message
news:2547FAF9-8BB6-407C-AFCA-E5D2F9082EB0@.microsoft.com...
> Hi,
> I've got several production DBs that I want to change the recovery model
> on from simple to full. Are there any caveats I need to be aware of in
> making this change? Or is it just a matter of selecting the 'Full'
> recovery
> model under the 'Options' tab of the DBs Properties and hitting the 'OK'
> button? Thanks.

Monday, February 20, 2012

Question on association models: MINIMUM_ITEMSET_SIZE

I've been experimenting with the algorithm parameters for a market basket association model. The default MINIMUM_ITEMSET_SIZE is 1. This doesn't seem to make sense: what is the point of a single-member itemset? However changing the value to 2 substantially reduces the proportion of good recommendations obtained (which I'm testing via a holdout approach).

So I'm obviously misunderstanding what the parameter means. Can someone explain it please, and also explain the observation above)?

Every item in a dataset is an "itemset", so there are always "1-itemsets", that's just the list of items and their counts. MINIMUM_ITEMSET_SIZE is really only there for those who are only interested in itemsets larger than 2. Of course, the process of building the model still has to create all the smaller itemsets to get to the larger ones, but it then discards them for prediction and reporting purposes.