Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2696 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.4213. Min Range: -5.4213

Max Act: 5.4213. Min Act: -0.1700

Data Index: 7928932 (Open Web Text)

Max Activating Token Index: 162

Text #1

Max Range: 5.4213. Min Range: -5.4213

Max Act: 5.3168. Min Act: -0.1700

Data Index: 4423743 (Open Web Text)

Max Activating Token Index: 303

Text #2

Max Range: 5.4213. Min Range: -5.4213

Max Act: 5.2949. Min Act: -0.1700

Data Index: 3405393 (Open Web Text)

Max Activating Token Index: 425

Text #3

Max Range: 5.4213. Min Range: -5.4213

Max Act: 5.1821. Min Act: -0.1700

Data Index: 4463350 (Open Web Text)

Max Activating Token Index: 545

Text #4

Max Range: 5.4213. Min Range: -5.4213

Max Act: 5.1626. Min Act: -0.1700

Data Index: 1697063 (Open Web Text)

Max Activating Token Index: 973

Text #5

Max Range: 5.4213. Min Range: -5.4213

Max Act: 5.1728. Min Act: -0.1700

Data Index: 3678857 (Open Web Text)

Max Activating Token Index: 184

Text #6

Max Range: 5.4213. Min Range: -5.4213

Max Act: 5.1476. Min Act: -0.1700

Data Index: 4015881 (Open Web Text)

Max Activating Token Index: 363

Text #7

Max Range: 5.4213. Min Range: -5.4213

Max Act: 5.0690. Min Act: -0.1700

Data Index: 4355289 (Open Web Text)

Max Activating Token Index: 415

Text #8

Max Range: 5.4213. Min Range: -5.4213

Max Act: 5.0860. Min Act: -0.1700

Data Index: 8235988 (Open Web Text)

Max Activating Token Index: 420

Text #9

Max Range: 5.4213. Min Range: -5.4213

Max Act: 5.0895. Min Act: -0.1700

Data Index: 5794270 (Open Web Text)

Max Activating Token Index: 851

Text #10

Max Range: 5.4213. Min Range: -5.4213

Max Act: 5.0816. Min Act: -0.1700

Data Index: 6947238 (Open Web Text)

Max Activating Token Index: 484

Text #11

Max Range: 5.4213. Min Range: -5.4213

Max Act: 5.0694. Min Act: -0.1700

Data Index: 7240428 (Open Web Text)

Max Activating Token Index: 198

Text #12

Max Range: 5.4213. Min Range: -5.4213

Max Act: 5.0276. Min Act: -0.1700

Data Index: 769460 (Open Web Text)

Max Activating Token Index: 491

Text #13

Max Range: 5.4213. Min Range: -5.4213

Max Act: 4.9967. Min Act: -0.1700

Data Index: 6621875 (Open Web Text)

Max Activating Token Index: 667

Text #14

Max Range: 5.4213. Min Range: -5.4213

Max Act: 4.9621. Min Act: -0.1700

Data Index: 3302952 (Open Web Text)

Max Activating Token Index: 660

Text #15

Max Range: 5.4213. Min Range: -5.4213

Max Act: 4.9476. Min Act: -0.1700

Data Index: 2113434 (Open Web Text)

Max Activating Token Index: 521

Text #16

Max Range: 5.4213. Min Range: -5.4213

Max Act: 4.9313. Min Act: -0.1700

Data Index: 3645555 (Open Web Text)

Max Activating Token Index: 955

Text #17

Max Range: 5.4213. Min Range: -5.4213

Max Act: 4.9382. Min Act: -0.1700

Data Index: 441914 (Open Web Text)

Max Activating Token Index: 463

Text #18

Max Range: 5.4213. Min Range: -5.4213

Max Act: 4.8847. Min Act: -0.1700

Data Index: 3294825 (Open Web Text)

Max Activating Token Index: 445

Text #19

Max Range: 5.4213. Min Range: -5.4213

Max Act: 4.9032. Min Act: -0.1700

Data Index: 156848 (Open Web Text)

Max Activating Token Index: 553