< Prev | Home | Model | Random | Next >

Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 5089 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')



Text #0

Max Range: 1.3997. Min Range: -1.3997

Max Act: 1.3997. Min Act: -0.0000

Data Index: 1015524 (The Pile)

Max Activating Token Index: 343

Click toggle to see full text

Truncated

Full Text #0


Text #1

Max Range: 1.3997. Min Range: -1.3997

Max Act: 1.3436. Min Act: -0.0000

Data Index: 217121 (The Pile)

Max Activating Token Index: 761

Click toggle to see full text

Truncated

Full Text #1


Text #2

Max Range: 1.3997. Min Range: -1.3997

Max Act: 1.1793. Min Act: -0.0000

Data Index: 813060 (The Pile)

Max Activating Token Index: 35

Click toggle to see full text

Truncated

Full Text #2


Text #3

Max Range: 1.3997. Min Range: -1.3997

Max Act: 1.1992. Min Act: -0.0000

Data Index: 1687963 (The Pile)

Max Activating Token Index: 509

Click toggle to see full text

Truncated

Full Text #3


Text #4

Max Range: 1.3997. Min Range: -1.3997

Max Act: 1.0267. Min Act: -0.0000

Data Index: 914675 (The Pile)

Max Activating Token Index: 600

Click toggle to see full text

Truncated

Full Text #4


Text #5

Max Range: 1.3997. Min Range: -1.3997

Max Act: 1.1531. Min Act: -0.0000

Data Index: 1850301 (The Pile)

Max Activating Token Index: 932

Click toggle to see full text

Truncated

Full Text #5


Text #6

Max Range: 1.3997. Min Range: -1.3997

Max Act: 1.0979. Min Act: -0.0000

Data Index: 1496093 (The Pile)

Max Activating Token Index: 826

Click toggle to see full text

Truncated

Full Text #6


Text #7

Max Range: 1.3997. Min Range: -1.3997

Max Act: 1.0315. Min Act: -0.0000

Data Index: 485478 (The Pile)

Max Activating Token Index: 616

Click toggle to see full text

Truncated

Full Text #7


Text #8

Max Range: 1.3997. Min Range: -1.3997

Max Act: 1.1168. Min Act: -0.0000

Data Index: 186206 (The Pile)

Max Activating Token Index: 866

Click toggle to see full text

Truncated

Full Text #8


Text #9

Max Range: 1.3997. Min Range: -1.3997

Max Act: 1.1374. Min Act: -0.0000

Data Index: 932405 (The Pile)

Max Activating Token Index: 187

Click toggle to see full text

Truncated

Full Text #9


Text #10

Max Range: 1.3997. Min Range: -1.3997

Max Act: 1.1680. Min Act: -0.0000

Data Index: 892678 (The Pile)

Max Activating Token Index: 967

Click toggle to see full text

Truncated

Full Text #10


Text #11

Max Range: 1.3997. Min Range: -1.3997

Max Act: 1.0352. Min Act: -0.0000

Data Index: 1776923 (The Pile)

Max Activating Token Index: 109

Click toggle to see full text

Truncated

Full Text #11


Text #12

Max Range: 1.3997. Min Range: -1.3997

Max Act: 1.1602. Min Act: -0.0000

Data Index: 1470513 (The Pile)

Max Activating Token Index: 102

Click toggle to see full text

Truncated

Full Text #12


Text #13

Max Range: 1.3997. Min Range: -1.3997

Max Act: 1.1135. Min Act: -0.0000

Data Index: 1350863 (The Pile)

Max Activating Token Index: 388

Click toggle to see full text

Truncated

Full Text #13


Text #14

Max Range: 1.3997. Min Range: -1.3997

Max Act: 1.1470. Min Act: -0.0000

Data Index: 81618 (The Pile)

Max Activating Token Index: 251

Click toggle to see full text

Truncated

Full Text #14


Text #15

Max Range: 1.3997. Min Range: -1.3997

Max Act: 1.0966. Min Act: -0.0000

Data Index: 1840509 (The Pile)

Max Activating Token Index: 232

Click toggle to see full text

Truncated

Full Text #15


Text #16

Max Range: 1.3997. Min Range: -1.3997

Max Act: 0.9731. Min Act: -0.0000

Data Index: 164864 (The Pile)

Max Activating Token Index: 849

Click toggle to see full text

Truncated

Full Text #16


Text #17

Max Range: 1.3997. Min Range: -1.3997

Max Act: 1.0222. Min Act: -0.0000

Data Index: 543666 (The Pile)

Max Activating Token Index: 354

Click toggle to see full text

Truncated

Full Text #17


Text #18

Max Range: 1.3997. Min Range: -1.3997

Max Act: 0.9762. Min Act: -0.0000

Data Index: 1644276 (The Pile)

Max Activating Token Index: 426

Click toggle to see full text

Truncated

Full Text #18


Text #19

Max Range: 1.3997. Min Range: -1.3997

Max Act: 0.9718. Min Act: -0.0000

Data Index: 1967191 (The Pile)

Max Activating Token Index: 24

Click toggle to see full text

Truncated

Full Text #19