Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1915 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4302. Min Range: -3.4302

Max Act: 3.4192. Min Act: -0.1700

Data Index: 8217810 (Open Web Text)

Max Activating Token Index: 102

Text #1

Max Range: 3.4302. Min Range: -3.4302

Max Act: 3.4302. Min Act: -0.1700

Data Index: 7874055 (Open Web Text)

Max Activating Token Index: 738

Text #2

Max Range: 3.4302. Min Range: -3.4302

Max Act: 3.3851. Min Act: -0.1700

Data Index: 4650946 (Open Web Text)

Max Activating Token Index: 680

Text #3

Max Range: 3.4302. Min Range: -3.4302

Max Act: 3.3934. Min Act: -0.1700

Data Index: 6767636 (Open Web Text)

Max Activating Token Index: 512

Text #4

Max Range: 3.4302. Min Range: -3.4302

Max Act: 3.3926. Min Act: -0.1700

Data Index: 8605396 (Open Web Text)

Max Activating Token Index: 520

Text #5

Max Range: 3.4302. Min Range: -3.4302

Max Act: 3.3873. Min Act: -0.1700

Data Index: 2711040 (Open Web Text)

Max Activating Token Index: 173

Text #6

Max Range: 3.4302. Min Range: -3.4302

Max Act: 3.3502. Min Act: -0.1700

Data Index: 6783265 (Open Web Text)

Max Activating Token Index: 810

Text #7

Max Range: 3.4302. Min Range: -3.4302

Max Act: 3.3459. Min Act: -0.1700

Data Index: 6530009 (Open Web Text)

Max Activating Token Index: 1008

Text #8

Max Range: 3.4302. Min Range: -3.4302

Max Act: 3.3593. Min Act: -0.1700

Data Index: 7506489 (Open Web Text)

Max Activating Token Index: 364

Text #9

Max Range: 3.4302. Min Range: -3.4302

Max Act: 3.3560. Min Act: -0.1700

Data Index: 7330850 (Open Web Text)

Max Activating Token Index: 559

Text #10

Max Range: 3.4302. Min Range: -3.4302

Max Act: 3.3406. Min Act: -0.1700

Data Index: 3613884 (Open Web Text)

Max Activating Token Index: 710

Text #11

Max Range: 3.4302. Min Range: -3.4302

Max Act: 3.3339. Min Act: -0.1700

Data Index: 6177563 (Open Web Text)

Max Activating Token Index: 860

Text #12

Max Range: 3.4302. Min Range: -3.4302

Max Act: 3.3557. Min Act: -0.1700

Data Index: 5450825 (Open Web Text)

Max Activating Token Index: 730

Text #13

Max Range: 3.4302. Min Range: -3.4302

Max Act: 3.3313. Min Act: -0.1700

Data Index: 4962687 (Open Web Text)

Max Activating Token Index: 311

Text #14

Max Range: 3.4302. Min Range: -3.4302

Max Act: 3.3298. Min Act: -0.1700

Data Index: 1213990 (Open Web Text)

Max Activating Token Index: 63

Text #15

Max Range: 3.4302. Min Range: -3.4302

Max Act: 3.2993. Min Act: -0.1700

Data Index: 1528502 (Open Web Text)

Max Activating Token Index: 889

Text #16

Max Range: 3.4302. Min Range: -3.4302

Max Act: 3.3166. Min Act: -0.1700

Data Index: 4570772 (Open Web Text)

Max Activating Token Index: 332

Text #17

Max Range: 3.4302. Min Range: -3.4302

Max Act: 3.3093. Min Act: -0.1700

Data Index: 2193425 (Open Web Text)

Max Activating Token Index: 940

Text #18

Max Range: 3.4302. Min Range: -3.4302

Max Act: 3.3042. Min Act: -0.1700

Data Index: 2218176 (Open Web Text)

Max Activating Token Index: 159

Text #19

Max Range: 3.4302. Min Range: -3.4302

Max Act: 3.2884. Min Act: -0.1700

Data Index: 339714 (Open Web Text)

Max Activating Token Index: 425