Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 747 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0371. Min Range: -3.0371

Max Act: 3.0371. Min Act: -0.1700

Data Index: 4033911 (Open Web Text)

Max Activating Token Index: 721

Text #1

Max Range: 3.0371. Min Range: -3.0371

Max Act: 2.9856. Min Act: -0.1700

Data Index: 6055895 (Open Web Text)

Max Activating Token Index: 430

Text #2

Max Range: 3.0371. Min Range: -3.0371

Max Act: 2.9800. Min Act: -0.1700

Data Index: 7052757 (Open Web Text)

Max Activating Token Index: 477

Text #3

Max Range: 3.0371. Min Range: -3.0371

Max Act: 2.9208. Min Act: -0.1700

Data Index: 850247 (Open Web Text)

Max Activating Token Index: 412

Text #4

Max Range: 3.0371. Min Range: -3.0371

Max Act: 2.9196. Min Act: -0.1700

Data Index: 4477102 (Open Web Text)

Max Activating Token Index: 408

Text #5

Max Range: 3.0371. Min Range: -3.0371

Max Act: 2.9354. Min Act: -0.1700

Data Index: 6805374 (Open Web Text)

Max Activating Token Index: 848

Text #6

Max Range: 3.0371. Min Range: -3.0371

Max Act: 2.9016. Min Act: -0.1700

Data Index: 5581890 (Open Web Text)

Max Activating Token Index: 582

Text #7

Max Range: 3.0371. Min Range: -3.0371

Max Act: 2.8893. Min Act: -0.1700

Data Index: 4362054 (Open Web Text)

Max Activating Token Index: 314

Text #8

Max Range: 3.0371. Min Range: -3.0371

Max Act: 2.8631. Min Act: -0.1700

Data Index: 5819657 (Open Web Text)

Max Activating Token Index: 291

Text #9

Max Range: 3.0371. Min Range: -3.0371

Max Act: 2.8899. Min Act: -0.1700

Data Index: 2877251 (Open Web Text)

Max Activating Token Index: 484

Text #10

Max Range: 3.0371. Min Range: -3.0371

Max Act: 2.8913. Min Act: -0.1700

Data Index: 900444 (Open Web Text)

Max Activating Token Index: 157

Text #11

Max Range: 3.0371. Min Range: -3.0371

Max Act: 2.8430. Min Act: -0.1700

Data Index: 3558218 (Open Web Text)

Max Activating Token Index: 748

Text #12

Max Range: 3.0371. Min Range: -3.0371

Max Act: 2.8270. Min Act: -0.1700

Data Index: 6168581 (Open Web Text)

Max Activating Token Index: 244

Text #13

Max Range: 3.0371. Min Range: -3.0371

Max Act: 2.8199. Min Act: -0.1700

Data Index: 2993111 (Open Web Text)

Max Activating Token Index: 657

Text #14

Max Range: 3.0371. Min Range: -3.0371

Max Act: 2.8297. Min Act: -0.1700

Data Index: 4777151 (Open Web Text)

Max Activating Token Index: 408

Text #15

Max Range: 3.0371. Min Range: -3.0371

Max Act: 2.8230. Min Act: -0.1700

Data Index: 1747163 (Open Web Text)

Max Activating Token Index: 552

Text #16

Max Range: 3.0371. Min Range: -3.0371

Max Act: 2.8120. Min Act: -0.1700

Data Index: 969228 (Open Web Text)

Max Activating Token Index: 967

Text #17

Max Range: 3.0371. Min Range: -3.0371

Max Act: 2.7792. Min Act: -0.1700

Data Index: 3373658 (Open Web Text)

Max Activating Token Index: 749

Text #18

Max Range: 3.0371. Min Range: -3.0371

Max Act: 2.7924. Min Act: -0.1700

Data Index: 1717881 (Open Web Text)

Max Activating Token Index: 657

Text #19

Max Range: 3.0371. Min Range: -3.0371

Max Act: 2.8130. Min Act: -0.1700

Data Index: 3683149 (Open Web Text)

Max Activating Token Index: 227