Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 399 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.4920. Min Range: -5.4920

Max Act: 5.4920. Min Act: -0.1700

Data Index: 1442855 (Open Web Text)

Max Activating Token Index: 6

Text #1

Max Range: 5.4920. Min Range: -5.4920

Max Act: 5.4422. Min Act: -0.1700

Data Index: 4048770 (Open Web Text)

Max Activating Token Index: 4

Text #2

Max Range: 5.4920. Min Range: -5.4920

Max Act: 5.4110. Min Act: -0.1700

Data Index: 8766729 (Open Web Text)

Max Activating Token Index: 6

Text #3

Max Range: 5.4920. Min Range: -5.4920

Max Act: 5.4147. Min Act: -0.1700

Data Index: 116902 (Open Web Text)

Max Activating Token Index: 4

Text #4

Max Range: 5.4920. Min Range: -5.4920

Max Act: 5.4203. Min Act: -0.1700

Data Index: 3246506 (Open Web Text)

Max Activating Token Index: 4

Text #5

Max Range: 5.4920. Min Range: -5.4920

Max Act: 5.4255. Min Act: -0.1700

Data Index: 7487125 (Open Web Text)

Max Activating Token Index: 6

Text #6

Max Range: 5.4920. Min Range: -5.4920

Max Act: 5.4265. Min Act: -0.1700

Data Index: 1046685 (Open Web Text)

Max Activating Token Index: 7

Text #7

Max Range: 5.4920. Min Range: -5.4920

Max Act: 5.4215. Min Act: -0.1700

Data Index: 7441555 (Open Web Text)

Max Activating Token Index: 7

Text #8

Max Range: 5.4920. Min Range: -5.4920

Max Act: 5.3891. Min Act: -0.1700

Data Index: 4807700 (Open Web Text)

Max Activating Token Index: 4

Text #9

Max Range: 5.4920. Min Range: -5.4920

Max Act: 5.3890. Min Act: -0.1700

Data Index: 5306205 (Open Web Text)

Max Activating Token Index: 5

Text #10

Max Range: 5.4920. Min Range: -5.4920

Max Act: 5.3659. Min Act: -0.1700

Data Index: 6028089 (Open Web Text)

Max Activating Token Index: 5

Text #11

Max Range: 5.4920. Min Range: -5.4920

Max Act: 5.3827. Min Act: -0.1700

Data Index: 4696856 (Open Web Text)

Max Activating Token Index: 5

Text #12

Max Range: 5.4920. Min Range: -5.4920

Max Act: 5.3949. Min Act: -0.1700

Data Index: 8521627 (Open Web Text)

Max Activating Token Index: 5

Text #13

Max Range: 5.4920. Min Range: -5.4920

Max Act: 5.3682. Min Act: -0.1700

Data Index: 8716889 (Open Web Text)

Max Activating Token Index: 4

Text #14

Max Range: 5.4920. Min Range: -5.4920

Max Act: 5.3507. Min Act: -0.1700

Data Index: 8035640 (Open Web Text)

Max Activating Token Index: 6

Text #15

Max Range: 5.4920. Min Range: -5.4920

Max Act: 5.3438. Min Act: -0.1700

Data Index: 3924115 (Open Web Text)

Max Activating Token Index: 6

Text #16

Max Range: 5.4920. Min Range: -5.4920

Max Act: 5.3355. Min Act: -0.1700

Data Index: 7201368 (Open Web Text)

Max Activating Token Index: 4

Text #17

Max Range: 5.4920. Min Range: -5.4920

Max Act: 5.3475. Min Act: -0.1700

Data Index: 6790545 (Open Web Text)

Max Activating Token Index: 10

Text #18

Max Range: 5.4920. Min Range: -5.4920

Max Act: 5.3399. Min Act: -0.1700

Data Index: 6194642 (Open Web Text)

Max Activating Token Index: 6

Text #19

Max Range: 5.4920. Min Range: -5.4920

Max Act: 5.3604. Min Act: -0.1700

Data Index: 347345 (Open Web Text)

Max Activating Token Index: 9