Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2794 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.4409. Min Range: -2.4409

Max Act: 2.4409. Min Act: -0.1700

Data Index: 8322537 (Open Web Text)

Max Activating Token Index: 5

Text #1

Max Range: 2.4409. Min Range: -2.4409

Max Act: 2.3751. Min Act: -0.1700

Data Index: 5942372 (Open Web Text)

Max Activating Token Index: 76

Text #2

Max Range: 2.4409. Min Range: -2.4409

Max Act: 2.3593. Min Act: -0.1700

Data Index: 5405927 (Open Web Text)

Max Activating Token Index: 322

Text #3

Max Range: 2.4409. Min Range: -2.4409

Max Act: 2.3038. Min Act: -0.1700

Data Index: 680652 (Open Web Text)

Max Activating Token Index: 41

Text #4

Max Range: 2.4409. Min Range: -2.4409

Max Act: 2.2866. Min Act: -0.1700

Data Index: 3165169 (Open Web Text)

Max Activating Token Index: 80

Text #5

Max Range: 2.4409. Min Range: -2.4409

Max Act: 2.2818. Min Act: -0.1700

Data Index: 4331839 (Open Web Text)

Max Activating Token Index: 22

Text #6

Max Range: 2.4409. Min Range: -2.4409

Max Act: 2.2834. Min Act: -0.1700

Data Index: 4215163 (Open Web Text)

Max Activating Token Index: 99

Text #7

Max Range: 2.4409. Min Range: -2.4409

Max Act: 2.2685. Min Act: -0.1700

Data Index: 4442172 (Open Web Text)

Max Activating Token Index: 108

Text #8

Max Range: 2.4409. Min Range: -2.4409

Max Act: 2.2719. Min Act: -0.1700

Data Index: 3341730 (Open Web Text)

Max Activating Token Index: 244

Text #9

Max Range: 2.4409. Min Range: -2.4409

Max Act: 2.2642. Min Act: -0.1700

Data Index: 3763100 (Open Web Text)

Max Activating Token Index: 7

Text #10

Max Range: 2.4409. Min Range: -2.4409

Max Act: 2.2660. Min Act: -0.1700

Data Index: 6439029 (Open Web Text)

Max Activating Token Index: 122

Text #11

Max Range: 2.4409. Min Range: -2.4409

Max Act: 2.2611. Min Act: -0.1700

Data Index: 6112586 (Open Web Text)

Max Activating Token Index: 5

Text #12

Max Range: 2.4409. Min Range: -2.4409

Max Act: 2.2622. Min Act: -0.1700

Data Index: 7075568 (Open Web Text)

Max Activating Token Index: 12

Text #13

Max Range: 2.4409. Min Range: -2.4409

Max Act: 2.2574. Min Act: -0.1700

Data Index: 7639839 (Open Web Text)

Max Activating Token Index: 27

Text #14

Max Range: 2.4409. Min Range: -2.4409

Max Act: 2.2666. Min Act: -0.1700

Data Index: 7828225 (Open Web Text)

Max Activating Token Index: 19

Text #15

Max Range: 2.4409. Min Range: -2.4409

Max Act: 2.2711. Min Act: -0.1700

Data Index: 5333919 (Open Web Text)

Max Activating Token Index: 3

Text #16

Max Range: 2.4409. Min Range: -2.4409

Max Act: 2.2581. Min Act: -0.1700

Data Index: 4315098 (Open Web Text)

Max Activating Token Index: 4

Text #17

Max Range: 2.4409. Min Range: -2.4409

Max Act: 2.2442. Min Act: -0.1700

Data Index: 1791207 (Open Web Text)

Max Activating Token Index: 15

Text #18

Max Range: 2.4409. Min Range: -2.4409

Max Act: 2.2508. Min Act: -0.1700

Data Index: 5087292 (Open Web Text)

Max Activating Token Index: 2

Text #19

Max Range: 2.4409. Min Range: -2.4409

Max Act: 2.2465. Min Act: -0.1700

Data Index: 1231765 (Open Web Text)

Max Activating Token Index: 7