Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 262 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.7953. Min Range: -4.7953

Max Act: 4.7953. Min Act: -0.1700

Data Index: 8470742 (Open Web Text)

Max Activating Token Index: 46

Text #1

Max Range: 4.7953. Min Range: -4.7953

Max Act: 4.7556. Min Act: -0.1700

Data Index: 2481464 (Open Web Text)

Max Activating Token Index: 152

Text #2

Max Range: 4.7953. Min Range: -4.7953

Max Act: 4.6953. Min Act: -0.1700

Data Index: 6097069 (Open Web Text)

Max Activating Token Index: 542

Text #3

Max Range: 4.7953. Min Range: -4.7953

Max Act: 4.6734. Min Act: -0.1700

Data Index: 5327097 (Open Web Text)

Max Activating Token Index: 80

Text #4

Max Range: 4.7953. Min Range: -4.7953

Max Act: 4.6855. Min Act: -0.1700

Data Index: 203757 (Open Web Text)

Max Activating Token Index: 838

Text #5

Max Range: 4.7953. Min Range: -4.7953

Max Act: 4.5846. Min Act: -0.1700

Data Index: 2841755 (Open Web Text)

Max Activating Token Index: 226

Text #6

Max Range: 4.7953. Min Range: -4.7953

Max Act: 4.5738. Min Act: -0.1700

Data Index: 1726172 (Open Web Text)

Max Activating Token Index: 479

Text #7

Max Range: 4.7953. Min Range: -4.7953

Max Act: 4.5144. Min Act: -0.1700

Data Index: 287846 (Open Web Text)

Max Activating Token Index: 989

Text #8

Max Range: 4.7953. Min Range: -4.7953

Max Act: 4.4807. Min Act: -0.1700

Data Index: 5917046 (Open Web Text)

Max Activating Token Index: 57

Text #9

Max Range: 4.7953. Min Range: -4.7953

Max Act: 4.4963. Min Act: -0.1700

Data Index: 8054556 (Open Web Text)

Max Activating Token Index: 47

Text #10

Max Range: 4.7953. Min Range: -4.7953

Max Act: 4.4830. Min Act: -0.1700

Data Index: 7395958 (Open Web Text)

Max Activating Token Index: 909

Text #11

Max Range: 4.7953. Min Range: -4.7953

Max Act: 4.5121. Min Act: -0.1700

Data Index: 287845 (Open Web Text)

Max Activating Token Index: 529

Text #12

Max Range: 4.7953. Min Range: -4.7953

Max Act: 4.4607. Min Act: -0.1700

Data Index: 6288294 (Open Web Text)

Max Activating Token Index: 63

Text #13

Max Range: 4.7953. Min Range: -4.7953

Max Act: 4.4612. Min Act: -0.1700

Data Index: 6488070 (Open Web Text)

Max Activating Token Index: 624

Text #14

Max Range: 4.7953. Min Range: -4.7953

Max Act: 4.3687. Min Act: -0.1700

Data Index: 1889196 (Open Web Text)

Max Activating Token Index: 824

Text #15

Max Range: 4.7953. Min Range: -4.7953

Max Act: 4.4295. Min Act: -0.1700

Data Index: 1589025 (Open Web Text)

Max Activating Token Index: 102

Text #16

Max Range: 4.7953. Min Range: -4.7953

Max Act: 4.4371. Min Act: -0.1700

Data Index: 7392451 (Open Web Text)

Max Activating Token Index: 651

Text #17

Max Range: 4.7953. Min Range: -4.7953

Max Act: 4.3980. Min Act: -0.1700

Data Index: 3243388 (Open Web Text)

Max Activating Token Index: 605

Text #18

Max Range: 4.7953. Min Range: -4.7953

Max Act: 4.4280. Min Act: -0.1700

Data Index: 693296 (Open Web Text)

Max Activating Token Index: 932

Text #19

Max Range: 4.7953. Min Range: -4.7953

Max Act: 4.3587. Min Act: -0.1700

Data Index: 2532842 (Open Web Text)

Max Activating Token Index: 469